Editing Action

An edit action is an operation performed on a stream of data. The simplest case is invalidation: setting all affected data to explicitly missing values.

The action type is selected with the enumeration value Type:

Invalidate

Invalid data, pressing the presence of it but setting it to explicitly missing values. This causes the data to show up as missing, but without introducing a gap indicating the absence of acquisition.

Selection: The data selection to invalidate.
PreserveType: A boolean indicating if the invalidation preserves the type (e.g. real number) or if it sets it to a general undefined indicator. Defaults to enabled so the type is preserved, if possible.

Contaminate or Contam

Apply contamination to all data in the stream. This sets a contamination flag indicating that the data are not representative of ambient conditions and should not be averaged into final products.

Origin: The name of the source of contamination. This is part of the final flag name and defaults to "Mentor".
Description: The text description of the contamination source. This is recorded in the metadata about the flag and defaults to "Data flagged as contaminated by the station mentor".
Bits: The integer bits associated with the flag. This is used when generating a combined flags output and defaults to 0x2.

Uncontaminate or ClearContam

Remove all contamination flagging in the data stream. This undoes the effect of any prior source of contamination by cleaning all contamination related flags.

FlowCorrection or FlowCalibration

Apply a correction to a flow rate and any parameters derived from that flow rate. A flow correction is used when, for example, a calibration on a CLAP was later discovered to be incorrect. To apply a simple revision to an existing flow, use a unity (0, 1) calibration as the original, effectively disabling the normal calibration reversal.

Original: The original, incorrect, calibration of the flow rate.
Calibration: The final, true, calibration of the flow rate.
CombineCut: A boolean indicating if the accumulation handling should span cut sizes, defaulting to enabled.
Minimum: The real number minimum expected value of the uncalibrated (original reversed) input. This is used for root selection when reversing higher order polynomials.
Maximum: The real number maximum expected value of the uncalibrated (original reversed) input. This is used for root selection when reversing higher order polynomials.
Instrument: The name of the instrument to apply to. For example "A11".
Flow: When the instrument is not set, the data selection of the flow being corrected.
Selection: When the instrument is not set, the data selection of the flow derived values being corrected (e.g. absorptions).
Accumulator: When the instrument is not set, the data selection of any accumulated qualities derived from the flow (e.g. sample volume and length).
Spot: When the instrument is not set, the data selection of any associated total spot data.

Spot or SpotSize

Apply a correction for a filter absorption photometer spot size. This is used to correct for an inaccurately measured spot size on a single spot instrument (e.g. the PSAP-3W). If only a ratio is available, set the original spot size to one and the corrected to the ratio value.

Original: The original, incorrect, real number area of the spot in mm².
Corrected: The final, true, real number area of the spot in mm².
Instrument: The name of the instrument to apply to.
Selection: When the instrument is not set, the data selection of the flow derived values being corrected (e.g. absorptions).
Spot: When the instrument is not set, the data selection of any associated total spot data.
Length: When the instrument is not set, the data selection of any associated accumulated sample length.

MultiSpotSize, MultiSpot, or CLAPSpot

Apply a correction for a filter absorption photometer switched spot size. This is used to correct for an inaccurately measured spot size on a switched spot instrument (e.g. the CLAP). If only a ratio is available, set the original spot size to one and the corrected to the ratio value.

Original: An array of the original, incorrect, real number area of the spot in mm².
Corrected: An array of the final, true, real number area of the spot in mm².
Instrument: The name of the instrument to apply to.
Selection: When the instrument is not set, the data selection of the flow derived values being corrected (e.g. absorptions).
Spot: When the instrument is not set, the data selection of any associated total spot data.
Length: When the instrument is not set, the data selection of any associated accumulated sample length.

Polynomial, Poly, Cal, or Calibration

Apply a simple calibration polynomial to values in the data.

Calibration: The calibration to apply.
Selection: The data selection to apply the calibration to.

SetCut or Cut

Alter the logged cut size of data. This is used when the acquisition logged cut size does not match the true sampled cut size (e.g. the impactor was stuck). In general, the selection is used to select all data of the faulty cut size. If changing to a cut size that was not possibly active (i.e. in a non-switched system), the metadata application also needs to be enabled.

Selection: The data selection to alter.
Cut: The cut size to change to. For example, "PM10".
ApplyToMetadata: A boolean controlling if the size change also applies to metadata. This is not required if metadata for the target cut size already exists (e.g. a switched impactor system).

Serial or SetSerial

Set the serial number recorded in the metadata. This is used when the data recorded by the system has an inaccurate or incomplete serial number. The effect is purely an alteration of the metadata, no data values are changed.

SerialNumber: The usually integer serial number. Any data type is accepted here and will be directly copied to the metadata.
Instrument: The name of the instrument to apply to.
Selection: When the instrument is not set, the data selection to alter the serial number for.

AddFlag, Flag, or AddFlags

Add one or more flags to existing flags data values. This sets the specified flags in any selected data.

Flags: The flags to add.
Selection: The data selection of flags variables to add into.
Description: The text description of the added flag recorded into the metadata.
Bits: The integer bits associated with the flag.

RemoveFlag or RemoveFlags

Remove one or more flags from the data stream. This clears the specified flags in any selected data.

Flags: The flags to remove.
Selection: The data selection of flags variables to remove from.

RemoveMatchingFlags, RemoveAnyFlag, or RemoveAnyFlags

Remove any flags matching a regular expression pattern from the data stream. This will remove flags based on a pattern match, allowing for wildcard removal of multiple flags without explicitly specifying each one.

Flags: Any array of text patterns to match.
Selection: The data selection of flags variables to remove from.
CheckAll: A boolean that causes each individual flag to instance to be checked, instead of only removing the matching flags defined in the metadata. This is not normally required and is only used if the metadata is incomplete.

PolynomialInvert, PolyInvert, or InvertCal

Reverse or invert a polynomial, transforming the values to what they would be before the polynomial was applied.

Calibration: The calibration to invert.
Selection: The data selection to apply to.
Minimum: The real number minimum expected value of the uncalibrated input. This is used for root selection when reversing higher order polynomials.
Maximum: The real number maximum expected value of the uncalibrated input. This is used for root selection when reversing higher order polynomials.

Recalibrate

First reverse a polynomial then apply a new one.

Original: The original, incorrect, calibration to reverse.
Calibration: The final, true, calibration to apply.
Selection: The data selection to apply to.
Minimum: The real number minimum expected value of the uncalibrated (original reversed) input. This is used for root selection when reversing higher order polynomials.
Maximum: The real number maximum expected value of the uncalibrated (original reversed) input. This is used for root selection when reversing higher order polynomials.

MultiPolynomial, MultiPoly, or MultiCal or MultiCalibration

Apply a series of interpolated calibrations to the data. The result is an interpolation of the two bounding calibrations for any point. This allows for a calibration to be adjusted over time, allowing for an approximation for slow drift between "true" calibrations.

Selection

The data selection to apply to.

Calibrations

A map keyed, on absolute time, of the calibrations to apply.

Extrapolate

A boolean instructing the action to extrapolate at the endpoints, instead of using the endpoint calibration entirely when the time exceeds an endpoint.

Time

An enumeration controlling the time point each data value is evaluated with.

Middle or Center: Use the central time of the value.
Start or Begin: Use the start time of the value.
End: Use the end time of the value.

Interpolation

An enumeration specifying how the calibration results are interpolated together in time to produce the output.

Linear

Simple linear interpolation.

CubicSpline or CSpline

Cubic spline interpolation.

ClampStart: The real number slope of the interpolator at the first point. If set to undefined, a natural spline is used; meaning that the slope is set such that the second derivative is zero.
ClampEnd: The real number slope of the interpolator at the last point. If set to undefined, a natural spline is used; meaning that the slope is set such that the second derivative is zero.

Sin

Linear with a sin(x) based transfer function (concave).

SinPower: The real number power in the sinˣ. Larger values result in a steeper transition.

Cos

Linear with a cos(x) based transfer function (convex).

CosPower: The real number power in the cosˣ. Larger values result in a steeper transition.

Sigmoid

Linear with a sigmoid based transfer function (symmetric).

SigmoidSlope: The real number sharpness and shape of the sigmoid translation.

Wrap, Modular, or Modulus

Wrap values to within a constant range. For example, if a value exceeds the end it is wrapped around by the amount it exceeds it by from to the start. A common instance is a direction in degrees wrapped from zero to 360.

Selection: The data selection to apply wrapping to.
Start: The real number the lower start bound of the wrapping range.
End: The real number the upper end bound of the wrapping range.

Remove

Remove values entirely from the stream, creating a gap.

Selection: The data selection to remove.

Flavors

Alter the data identifier by adding and/or removing flavors from it.

Selection: The data selection before alteration.
ApplyToMetadata: A boolean controlling if alteration is also applied to metadata. When absent, the alteration is applied to metadata.
Add: An array of text flavor names to add to the identifier.
Remove: An array of text flavor names to remove from the identifier.

Unit or SetUnit

Alter the data identifier main components.

Selection: The data selection before alteration.
ApplyToMetadata: A boolean controlling if alteration is also applied to metadata. When absent, the alteration is applied to metadata.
Station: The text altered station identifier.
Archive: The text altered archive identifier.
Variable: The text altered variable identifier.
Flavors: An array of text altered identifier flavors.

UnitReplace or Translate

Alter the data identifier with substitutions extracted from regular expressions applied to the original. The regular expression captures are available in substations like "${1}" for the first (non-whole pattern) capture on the associated component.

Selection: The data selection before alteration.
ApplyToMetadata: A boolean controlling if alteration is also applied to metadata. When absent, the alteration is applied to metadata.
From/Station: The text regular expression applied to the original station identifier.
From/Archive: The text regular expression applied to the original archive identifier.
From/Variable: The text regular expression applied to the original variable identifier.
From/Flavor: The text regular expression applied to the original flavors identifiers.
To/Station: The text altered station identifier.
To/Archive: The text altered archive identifier.
To/Variable: The text altered variable identifier.
To/Flavor: The text altered identifier flavor.

Duplicate

Create a duplicate stream of data by copying an existing one then altering its unique identifier.

Selection: The data selection before alteration.
ApplyToMetadata: A boolean controlling if alteration is also applied to metadata. When absent, the alteration is applied to metadata.
Station: The text altered station identifier.
Archive: The text altered archive identifier.
Variable: The text altered variable identifier.
Flavors: An array of text altered identifier flavors.

DuplicateTranslate

Create a duplicate stream of data by copying an existing one then altering its unique identifier. This variant also provides substitutions extracted from regular expressions applied to the original. The regular expression captures are available in substations like "${1}" for the first (non-whole pattern) capture on the associated component.

Selection: The data selection before alteration.
ApplyToMetadata: A boolean controlling if alteration is also applied to metadata. When absent, the alteration is applied to metadata.
From/Station: The text regular expression applied to the original station identifier.
From/Archive: The text regular expression applied to the original archive identifier.
From/Variable: The text regular expression applied to the original variable identifier.
From/Flavor: The text regular expression applied to the original flavors identifiers.
To/Station: The text altered station identifier.
To/Archive: The text altered archive identifier.
To/Variable: The text altered variable identifier.
To/Flavor: The text altered identifier flavor.

Overlay or Set

Set a value directly by overlaying some contents at a specific path. If the overlay value is opaque then this is a simple set, while complex types are overlaid on any original contents.

Selection: The data selection to alter.
Path: The path within the original values to overlay at.
Value: The value contents to set the output to. Any time is allowed here and it is directly overlaid into the output.

Metadata or Meta or OverlayMeta or OverlayMetadata

Alter metadata by overlaying some contents at a specific path. If the overlay value is opaque then this is a simple set, while complex types are overlaid on any original contents.

Selection: The data selection to alter.
Path: The path within the original values to overlay at.
Value: The value contents to set the output to. Any time is allowed here and it is directly overlaid into the output.

Function, Math, or Arithmetic

Apply an arithmetic operation to the data. This allows for simple operations like adding one value to another one.

Selection

The data selection to alter. This also provides one of the inputs to the function being applied.

Inputs

The data selection of the other input. If multiple values are present, an unspecified valid one is selected.

Calibration

The calibration to applied to the pure input, before the function is applied.

Path

The path within the pure input values to read from.

Function

An enumeration specifying the function to apply to generate the output values.

Add or Plus: Add the input to the data (Selection + Calibration(Input)).
Subtract or Minus: Subtract the input from the data (Selection - Calibration(Input)).
Multiply or Times: Multiply the input and the data (Selection * Calibration(Input)).
Divide: Divide the data by the input (Selection / Calibration(Input)).
Invert or Inverse: Divide the input by the data, effectively inverting the data (Calibration(Input) / Selection).
Assign, Set, or Equal: Divide the input by the data, effectively inverting the data (Calibration(Input) / Selection).

ScriptDemultiplexer or Demultiplexer

Apply a script controlled segment demultiplexer to the data stream.

Code: The script code to execute to define the demultiplexer.

ScriptSegment, ScriptSegments, or Script

Apply a script processor for each segment in the data stream.

Code: The script code to execute for each time segment.
Selection: The data selection to apply the processor to. This serves as both the input and output to the processor if explicit ones are not defined.
Inputs: The data selection of explicit inputs.
Outputs: The data selection of explicit outputs.
ImplicitMetadata: A boolean controlling if metadata inputs are implicitly added to the segments. When absent, metadata are included.

ScriptValue or ScriptValues

Apply a script processor for each value in the data stream.

Code: The script code to execute for each value.
Selection: The data selection to use as inputs to the processor. All output values are placed into the data stream.

ScriptGeneralSegment or ScriptGeneralSegments

Execute a full script thread, with segments as the inputs to the data stream it sees.

Code: The script defining the execution thread. This normally includes a loop to process all inputs.
Selection: The data selection to apply to. This serves as both the input and output to the processor if explicit ones are not defined.
Inputs: The data selection of explicit inputs.
Outputs: The data selection of explicit outputs.
ImplicitMetadata: A boolean controlling if metadata inputs are implicitly added to the segments. When absent, metadata are included.

ScriptGeneralValue or ScriptGeneralValues

Execute a full script thread, with data values as the inputs to the data stream it sees.

Code: The script defining the execution thread. This normally includes a loop to process all inputs.
Selection: The data selection to use as inputs to the processor. All output values are placed into the data stream.

NOOP or None

No operation, effectively disabling the action.

Simple invalidation

/Type,"Invalidate"
/Selection/#0/Variable,"BsG_S11"

CLAP spot size correction

/Type,"MultiSpotSize"
/Instrument,"A11"
/Original/#0,19.81
/Original/#1,19.81
/Original/#2,19.81
/Original/#3,19.81
/Original/#4,19.81
/Original/#5,19.81
/Original/#6,19.81
/Original/#7,19.81
/Corrected/#0,21.2
/Corrected/#1,20.1
/Corrected/#2,19.5
/Corrected/#3,20.3
/Corrected/#4,19.9
/Corrected/#5,21.0
/Corrected/#6,20.9
/Corrected/#7,21.7