pycif.plugins.datastreams.fields.grib2_ecmwf — API reference

pycif.plugins.datastreams.fields.grib2_ecmwf — API reference#

Configuration reference: grib2_ecmwf plugin

pycif.plugins.datastreams.fields.grib2_ecmwf.fetch.fetch(ref_dir, ref_file, input_interval, target_dir, tracer=None, **kwargs)[source]#

Link the closest valid ECMWF GRIB files and build the date/file maps.

For each date spaced by tracer.file_freq across input_interval, calls find_valid_file() to locate the closest available GRIB file (and, when tracer.decumul is set, the surrounding file needed for decumulation), links the file into target_dir, and records the corresponding sub-interval: [dd, dd + file_freq] when decumulating or when tracer.valid_interval == "left", otherwise a symmetric interval centered on dd.

Parameters:

  • ref_dir – Date-format directory pattern where the original files are found.

  • ref_file – Date-format pattern for the original file names.

  • input_interval – List of two dates, the beginning and end of the period to fetch.

  • target_dir – Directory where links to the original files are created.

  • tracer – Tracer/component configuration (file_freq, cumul_length, decumul, valid_interval, delta_tolerance, etc.).

  • **kwargs – Unused.

Returns:

A tuple (list_files, list_dates) of dictionaries keyed by file date, mapping to the list of files and covered sub-intervals.

pycif.plugins.datastreams.fields.grib2_ecmwf.get_domain.make_regular_grid(lat, lon, jscan)[source]#

Build center/corner lon-lat grids for a regular lat-lon GRIB grid.

Parameters:

  • lat – 1D array of latitude centers, as read from the GRIB file.

  • lon – 1D array of longitude centers, as read from the GRIB file.

  • jscan – Value of the GRIB jScansPositively key; if 0, the latitude axis is flipped to increasing order.

Returns:

(zlonc, zlatc, zlon, zlat, nlon, nlat) where zlon/ zlat are the 2D meshgrid of cell centers, zlonc/zlatc the 2D meshgrid of cell corners, and nlon/nlat the grid dimensions.

Return type:

tuple

pycif.plugins.datastreams.fields.grib2_ecmwf.get_domain.make_gaussian_grid(lat, lon)[source]#

Build an unstructured cell-vertex grid for a reduced-Gaussian GRIB grid.

Groups consecutive grid points into blocks of constant latitude (each block corresponding to a Gaussian latitude ring), and computes the 4-vertex lon/lat corners of each cell from the spacing within its ring and the mid-points to the neighboring rings (poles are extended to +/-90 degrees).

Parameters:

  • lat – 1D array of latitude values, one per grid point (repeated within each latitude ring), as read from the GRIB file.

  • lon – 1D array of longitude values, one per grid point, as read from the GRIB file.

Returns:

(zlonc, zlatc, zlon, zlat, nlon, nlat) where zlonc/ zlatc are arrays of shape (4, ncell) giving the cell vertices, zlon/zlat are the cell centers reshaped to (1, ncell), nlon is the number of cells and nlat is 1 (the grid is unstructured).

Return type:

tuple

Raises:

CifValueError – If a latitude block is not constant, or if the longitude spacing within a block is not regular.

pycif.plugins.datastreams.fields.grib2_ecmwf.get_domain.make_octahedral_grid(lat, lon, file_corner)[source]#

Build an unstructured cell-vertex grid for an octahedral GRIB grid.

Reads pre-computed corner coordinates from a csv file (since octahedral grid geometry cannot be derived analytically from the center coordinates alone).

Parameters:

  • lat – 1D array of latitude centers, as read from the GRIB file.

  • lon – 1D array of longitude centers, as read from the GRIB file.

  • file_corner – Path to a csv file with one line per grid cell, each giving its 4 [lon, lat] corner pairs.

Returns:

(zlonc, zlatc, zlon, zlat, nlon, nlat) where zlonc/ zlatc are arrays of shape (4, ncell) giving the cell vertices, zlon/zlat are the cell centers reshaped to (1, ncell), nlon is the number of cells (number of lines in file_corner) and nlat is 1 (the grid is unstructured).

Return type:

tuple

pycif.plugins.datastreams.fields.grib2_ecmwf.get_domain.get_domain(ref_dir, ref_file, input_interval, target_dir, tracer=None)[source]#

Build the pyCIF domain from an ECMWF GRIB reference file.

Opens the domain reference file (the first fetched input file, or tracer.domain_file if given), determines the grid type via get_grid_type() ("octahedral", "reduced_gaussian" or "regular"), builds the corresponding center/corner grid, and reconstructs the hybrid vertical coefficients sigma_a/sigma_b from the GRIB pv key (cropped to the levels actually present via the hybrid key), or uses a single surface level if tracer.surface is set.

Parameters:

  • ref_dir – Unused directly (kept for interface consistency); reference files are taken from tracer.input_files.

  • ref_file – Unused directly (kept for interface consistency).

  • input_interval – Unused directly (kept for interface consistency).

  • target_dir – Unused directly (kept for interface consistency).

  • tracer – Tracer/component configuration; tracer.input_files, tracer.domain_file, tracer.filter_by_keys_dict and tracer.surface are used.

Returns:

The domain built from the reference file’s horizontal grid and vertical coefficients.

Return type:

pycif.utils.classes.domains.Domain

Raises:

  • CifFileNotFoundError – If the octahedral grid corner-coordinates file cannot be found.

  • CifValueError – If the grid type is not one of the supported types.

pycif.plugins.datastreams.fields.grib2_ecmwf.read.read(self, name, varnames, dates, files, interpol_flx=False, comp_type=None, tracer=None, **kwargs)[source]#

Read a variable from ECMWF GRIB files and load it into a pyCIF DataArray.

For each date, opens the associated GRIB file(s) via GribDataset, flips the latitude axis if the file’s scan direction requires it, flips the vertical axis for non-surface fields, decumulates the field (using the two straddling files fetched for that purpose) if tracer.decumul is set, and optionally expands surface pressure into a full 3D pressure (or pressure-thickness) field.

Parameters:

  • self – The field/tracer Plugin.

  • name – Name of the component (unused directly here).

  • varnames – Name of the GRIB variable to read.

  • dates – List of date entries (each a [date, ...]-like sequence) to extract, matching files.

  • files – List of file-group entries matching dates; each entry is itself a list of one (regular) or two (decumulation) file paths.

  • interpol_flx – Unused.

  • comp_type – Unused.

  • tracer – Tracer/component configuration (filter_by_keys_dict, surface, decumul, expand_psurf, pressure_thickness, domain, etc.).

  • **kwargs – Unused.

Returns:

A 4-dimensional (time, lev, lat, lon) array.

Return type:

xarray.DataArray

class pycif.plugins.datastreams.fields.grib2_ecmwf.utils.GribDataset(path: str | PathLike[str], read_keys: list[str] | None = None, filter_by_keys: dict[str, Any] | None = None)[source]#

Bases: object

Thread-safe wrapper around cfgrib to read GRIB files.

Exposes GRIB variables and attributes through a small, thread-safe API (all access to the underlying cfgrib/ecCodes handles is serialized via a module-level lock, since ecCodes is not thread-safe).

Parameters:

  • path – Path to the GRIB file.

  • read_keys – Additional GRIB keys to read, on top of the default ones read by cfgrib. Defaults to none.

  • filter_by_keys – Dict of GRIB keys/values passed to cfgrib.open_file to filter which messages are read. Defaults to none.

Raises:

CifFileNotFoundError – If path does not exist.

Example

>>> ds = GribDataset(
        "path/to/file.grib",
        read_keys=["isOctahedral"],
        filter_by_keys={"edition": 1}
    )
>>> ds["latitude"]  # get a variable
array([...])
>>> ds.get_attr("isOctahedral")  # get an attribute
1
close() → None[source]#

Explicitly release the underlying cfgrib handles under the grib lock, instead of leaving it to a garbage-collection pass on an arbitrary thread.

property variables: dict[str, Variable]#

Returns the variables mapping of the GRIB file

get_var(varname: str) → ndarray[source]#

Get one variable from the GRIB file as a Numpy array.

Parameters:

varname – Variable name.

Returns:

Variable data.

Return type:

np.ndarray

Raises:

CifKeyError – If the variable is not found in the GRIB file.

get_attr(attr: str, varname: str | None = None) → Any[source]#

Get an attribute from the GRIB file.

If varname is provided, looks for the attribute in that variable’s attributes; otherwise looks for it across all variables and returns the first one found.

Parameters:

  • attr – Attribute name to get (will be prefixed with "GRIB_" to match cfgrib attribute naming).

  • varname – Variable name to restrict the search to. Defaults to searching all variables.

Returns:

The attribute value.

Return type:

Any

Raises:

CifKeyError – If the attribute is not found.

pycif.plugins.datastreams.fields.grib2_ecmwf.utils.find_valid_file(file_format, dd, time_freq, ref_dir, ref_dir_next, ref_dir_previous, delta_tolerance=1, cumul_variable=False, cumul_length=12)[source]#

Find the GRIB file(s) closest to a requested date.

Lists files in ref_dir (plus, when the cumulation window straddles a month boundary, ref_dir_previous/ref_dir_next), parses their names against file_format to recover each file’s valid date and forecast lead time, and picks the file with the closest valid date at or before dd (preferring the smallest forecast lead time in case of a tie). If cumul_variable is set, also finds the next file after that date, matching the same forecast lead time, so the pair can be used to decumulate the variable.

Parameters:

  • file_format – Date-format pattern (with optional {di} forecast lead-time placeholder) used both to build expected file names and to parse actual file names into dates.

  • dd – The date to find a valid file for.

  • time_freq – Expected time step between files, used together with delta_tolerance to bound how far the closest file’s date may be from dd.

  • ref_dir – Directory to search for candidate files.

  • ref_dir_next – Directory to also search when the cumulation window crosses into the next month.

  • ref_dir_previous – Directory to also search when the cumulation window crosses into the previous month.

  • delta_tolerance – Tolerance, as a multiple of time_freq, allowed between dd and the closest file’s valid date.

  • cumul_variable – If True, also locate the next file needed to decumulate the variable.

  • cumul_length – Cumulation window length, in hours, used to decide whether ref_dir_previous/ref_dir_next must be searched.

Returns:

(files, dates), each a list with one entry (or two, when cumul_variable is set and a next file is found) giving the matched file path(s) and corresponding date(s) (re-expressed relative to dd).

Return type:

tuple

Raises:

  • CifFileNotFoundError – If no candidate file matches file_format, or if none is close enough to dd (within delta_tolerance * time_freq).

  • CifValueError – If decumulation is requested but no next file with a matching forecast lead time can be found.

pycif.plugins.datastreams.fields.grib2_ecmwf.utils.get_grid_type(domain_file: str | PathLike[str], filter_by_keys_dict: dict[str, Any] | None = None) → Literal['regular', 'octahedral', 'reduced_gaussian'][source]#

Determine the GRIB grid type of a reference file.

Parameters:

  • domain_file – Path to the GRIB file to inspect.

  • filter_by_keys_dict – Dict of GRIB keys/values used to select which message to read from the file.

Returns:

"octahedral" if the GRIB isOctahedral attribute is set, "regular" if the latitude spacing is constant, or "reduced_gaussian" otherwise.

pycif.plugins.datastreams.fields.grib2_ecmwf.utils.get_jscan(domain_file: str | PathLike[str], filter_by_keys_dict: dict[str, Any] | None = None) → int[source]#

Get the GRIB jScansPositively scan-direction attribute.

Parameters:

  • domain_file – Path to the GRIB file to inspect.

  • filter_by_keys_dict – Dict of GRIB keys/values used to select which message to read from the file.

Returns:

0 if latitudes scan from North to South (the default ECMWF convention), 1 if from South to North.

Return type:

int

Raises:

CifValueError – If the attribute is not defined in the file.