This component processes data and inserts it into the local archive. This usually takes the form of data transfer files uploaded by acquisition systems. It can also apply various forms of conversion of data files before the ingest. This differs from synchronization in that the incoming data are assumed be be newly created.

Usage

da.process.incoming [switches…​] [station…​]

Switches

--after=TIME

This sets the time to reprocess after. When set, this will process all files modified after the time. This also triggers a state reset when used. When set to undefined, all data are reprocessed.

Default: Last run time

Undefined values are permitted, to indicate that the option is set but not to a specific time.

--profile=STRING

This is the profile name to perform. Multiple profiles can be defined to perform different types of processing for a station. Consult the station configuration for available profiles.

Default: aerosol

--quiet[=BOOLEAN]

If set then no progress output is displayed and any confirmation prompts are bypassed.

Default: Disabled

--wait=NUMBER

This is the maximum time to wait for another upload process to complete, in seconds. If there are two concurrent processes for the same station and profile, the new one will wait at most this many seconds before giving up without uploading new data. If this is set to undefined, it will wait forever for the other process to complete.

Default: 30 seconds

This option only accepts numbers greater than or equal to 0. Undefined values are permitted, to indicate that the option is set but not to a specific value.

Arguments

station

This argument is used to specify the station used to look up variables that do not include a station as part of an archive read specification. The station is the three letter GAW station code of the location, such as BND. The argument accepts multiple stations specified as regular expressions and separated by : or ; or ,. For example BND,MLO and (BND|MLO) are two ways of selecting both the BND and MLO stations.

The special value allstations may also be specified to select all stations.

Examples

Process all data

This will process all incoming data.

da.process.incoming