This component provides a general purpose processor for data using the scripting interface. This processor can be configured to use demultiplexer, value, or segment semantics.
Usage
da.corr.script [switches…] [[station] variables times [archive]|[file]]
Switches
- --bypass=SELECTION
-
This option sets the values that bypass the processor. Bypass values may also be inputs, but should generally not be outputs.
Default: Anything not input or output
- --code=SCRIPT
-
This option sets the script code that is evaluated. When in general mode, the incoming data stream is available on the global variable "data". In processor mode, the variable contains the current value and "control" is stream control. In fanout mode the global variable "fanout" contains the fanout controller, which calls the handler with a control stream and fanout key as the two arguments. It expects a return of the handler with an optional output transformer.
- --input=SELECTION
-
This option set the values that are passed into the processor. Only values that this option matches are available as inputs
Default: Everything
- --mode=TYPE
-
Set the mode that the processing code is invoked as.
Default: Processor
The possible values are:
- --mode=fanout
-
Invoke the processing code as a fanout dispatcher. The fanout controller is available as the global "fanout" and should be called with the control handler before the script finishes.
- --mode=general
-
Invoke the processing code as a general script, expecting it to loop over the incoming data. The global "data" contains the value stream.
- --mode=processor
-
Invoke the processing code repeatedly as a handler. The global "data" contains the current value and "control" contains the stream controller.
- --output=SELECTION
-
This option set the values that are extracted from segments or not ignored by value processors.
Default: All segment inputs or all values produced
- --type=TYPE
-
Set the type of data stream provided.
Default: Segment
The possible values are:
- --type=segment
-
Provide data sequence segment values, containing multiple variables.
- --type=value
-
Provide individual data values, each representing a single un-flattened variable value.
Arguments
If no bare word input specification is supplied then data are read from standard input.
- station
-
This argument is used to specify the station used to look up variables that do not include a station as part of an archive read specification. The station is the three letter GAW station code of the location, such as
BND
. The argument accepts multiple stations specified as regular expressions and separated by:
or;
or,
. For exampleBND,MLO
and(BND|MLO)
are two ways of selecting both the BND and MLO stations.The special value
allstations
may also be specified to select all stations. - variables
-
This argument may be split into multiple actual program arguments. Each part consists of one or more variable specifications separated by
,
(commas). The station and archive used if none are explicitly given are defined in the other arguments or inferred from system defaults. For example simplyBsB_S11
selects the blue scattering from the S11 instrument and returns all cut sizes and metadata. If instead::BsB_S11:pm10
is used then only PM10 scattering is returned. That is, the selection specifies "any" station and archive (the defaults are set in the other arguments) then requires that it has the "PM10" flavor. A specification such asbnd:BsB_S11,thd:BsB_S11
allows for data selection from multiple stations.This may also be used to select a "record" type alias such as "S11a". In this mode all the variables defined in the alias are returned.
Finally the special value
everything
can be used to select all available data for the given station(s) and archive(s). - times
-
This argument may be split into multiple actual program arguments. This full list of time arguments defines a time bounds list that sets the range of data queried. The final resulting data are clipped to the given time bounds but any values that intersect them are returned.
- archive
-
This argument is used to specify the archive used to look up variables that do not include an archive as part of an archive read specification. The archive is the internal name, such as
raw
orclean_meta
. The argument accepts multiple archives specified as regular expressions and separated by:
or;
or,
. For exampleraw(_meta)?
selects both theraw
andraw_meta
archives.The special value
allarchives
may also be used to select all available archives. - file
-
This argument is used to specify the the file to read data from. If it is present and exists then data is read from the given file name instead of from standard input. Alternatively
-
(a single dash) may be used to explicitly specify standard input.
Examples
- Simple operation
-
This subtracts 0.25 from the BsG_S11 variable. This mode is only recommended for a single set of aligned data, since it forces all data into the same time segments.
da.corr.script --code='data.BsG_S11 = data.BsG_S11 - 0.25' bnd S11a 2015-05-01 2015-05-03
Or:
da.corr.script --code='data.BsG_S11 = data.BsG_S11 - 0.25' input_file.c3d
Or:
da.get bnd S11a 2015-05-01 2015-05-03 | da.corr.script --code='data.BsG_S11 = data.BsG_S11 - 0.25'
- Time alteration
-
This forces data into one minute alignment of starts and ends. Note that this can disrupt some averaging types.
da.corr.script --code='control.autorelease = 60; data.START = math.floor(data.START / 60) * 60; data.END = math.ceil(data.END / 60) * 60;' --type=value bnd S11a 2015-05-01 2015-05-03
- Data removal
-
This removes all data where the green scattering is less than 0.25.
da.corr.script --code='for v in data do if (v.BsG_S11 < 0.25) then data.erase(v); end end' --mode=general bnd S11a 2015-05-01 2015-05-03
- Fanout operation
-
This subtracts 0.25 from BsG_S11 while properly handling overlapping cut sizes, archives, and stations.
da.corr.script --code='fanout(function(control) return function(data) data.BsG_S11 = data.BsG_S11 - 0.25; end end)' --mode=fanout bnd S11a 2015-05-01 2015-05-03