Concatenation of observations from parsers standard/std

Concatenation of observations from parsers standard/std#

Description#

Standard measurement plugin — aggregate observations from one or more parsers.

This plugin is the primary entry point for pre-processing real observations. It reads raw data through one or more obsparser providers, concatenates the results into a single CIF-compatible data store (see the observation format for details), and feeds the observation vector.

Provider configuration#

Two modes are supported in the YAML configuration:

  1. Single provider — provider arguments are placed directly in the measurement paragraph:

    measurement:
  plugin:
    name: standard
  provider: my_obsparser
  ...   # obsparser-specific arguments

  2. Multiple providers — each provider is a named sub-paragraph under providers. The key names are arbitrary and do not affect processing:

    measurement:
  plugin:
    name: standard
  providers:
    surface_flask:
      provider: flask_obsparser
      ...
    aircraft:
      provider: aircraft_obsparser
      ...

For each provider, parse_multiple_files() is called with the tracer species name. Custom parsers may override this method for more complex retrieval chains (e.g. satellite data).

YAML arguments#

The following arguments are used to configure the plugin. pyCIF will return an exception at the initialization if mandatory arguments are not specified, or if any argument does not fit accepted values or type:

Optional arguments#

specname : str, optional

Species name to use if different than the one in the corresponding datavect paragraph

dump_type : str, optional, default “nc”

How to dump intermediate observation datastores

file_monitor : str, optional

Path to a pre-formatted monitor file.

providers : optional

List of providers and corresponding arguments to parse. See the documentation of obsparser objects for further details: here

provider : str, optional

Used only if providers is not specified. Can be used if only one provider is to be used. In that case, the provider arguments are directly specified at the same level.

YAML template#

Please find below a template for a YAML configuration:

 1measurements:
 2  plugin:
 3    name: standard
 4    version: std
 5    type: measurements
 6
 7  # Optional arguments
 8  specname: XXXXX  # str
 9  dump_type: XXXXX  # str
10  file_monitor: XXXXX  # str
11  providers: XXXXX  # any
12  provider: XXXXX  # str