pycif.plugins.transforms.system.fromcontrol — API reference

pycif.plugins.transforms.system.fromcontrol — API reference#

Configuration reference: fromcontrol plugin

pycif.plugins.transforms.system.fromcontrol.adjoint.adjoint(transform, inout_datastore, controlvect, obsvect, mapper, di, df, mode, runsubdir, workdir, onlyinit=False, **kwargs)[source]#

Accumulate model sensitivities into the control-vector gradient dx.

The adjoint of forward(): for each control tracer, takes the model sensitivity field adj_out (shape: time × level × lat × lon) from the output datastore and aggregates it back into the flattened control-vector gradient controlvect.dx.

The aggregation applies:

  1. Temporal aggregation — sensitivities are summed into the control-vector time periods via date-indexed lookup.

  2. Vertical aggregation — vmap2vaggreg() reduces the full 3-D sensitivity to the vertical resolution of the control vector (column, vpixels, or kbands).

  3. Horizontal projection — map2scale() maps the 2-D spatial sensitivity to the control-vector horizontal resolution (pixels, bands, regions, or global).

  4. Prior scaling — for scalar control variables the sensitivity is weighted by the prior physical input field value.

Non-control tracers and tracers absent from controlvect.datavect are silently skipped.

Parameters:

  • transform (Plugin) – fromcontrol transform instance.

  • inout_datastore (dict) – datastore; 'outputs' holds the model-space sensitivities (adj_out field).

  • controlvect – control vector plugin; dx is incremented in-place.

  • obsvect – observation vector plugin (unused).

  • mapper (dict) – transform mapper with output tracer metadata.

  • di (datetime) – sub-simulation start date.

  • df (datetime) – sub-simulation end date.

  • mode (str) – 'adj'.

  • runsubdir (str) – sub-simulation run directory.

  • workdir (str) – root working directory.

  • onlyinit (bool) – if True, return immediately (dry-run pass).

  • **kwargs – forwarded to tracer.read for prior scaling.

pycif.plugins.transforms.system.fromcontrol.forward.forward(transform, inout_datastore, controlvect, obsvect, mapper, di, df, mode, runsubdir, workdir, **kwargs)[source]#

Project the control vector onto physical model-input fields.

For each output tracer declared in the mapper:

  • Non-control tracers (iscontrol = False) — input files are read via tracer.read and stored verbatim.

  • Scalar control tracers (type = 'scalar', default) — the control-vector slice is unpacked to a map (scale2map()), re-indexed to the merged output date grid, and multiplied element-wise by the prior physical input field read from disk.

  • Physical control tracers (type = 'physical') — the control-vector slice is projected directly to physical space at the model domain resolution without reading any prior files.

In tangent-linear mode (mode = 'tl'), the same operations apply to controlvect.dx in parallel, producing an 'incr' field.

For ensemble runs (perturb_xb = True), delegates to forward_perturb() to handle all members in one I/O pass.

Parameters:

  • transform (Plugin) – fromcontrol transform instance.

  • inout_datastore (dict) – mutable datastore ('inputs', 'outputs').

  • controlvect – control vector plugin (x, dx).

  • obsvect – observation vector plugin (unused).

  • mapper (dict) – transform mapper with output tracer metadata.

  • di (datetime) – sub-simulation start date.

  • df (datetime) – sub-simulation end date.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • runsubdir (str) – sub-simulation run directory.

  • workdir (str) – root working directory.

  • **kwargs – forwarded to tracer.read.

pycif.plugins.transforms.system.fromcontrol.forward_perturb.forward_perturb(transform, inout_datastore, controlvect, obsvect, mapper, di, df, mode, runsubdir, workdir, **kwargs)[source]#

Ensemble-optimised forward projection for perturbation (EnSRF / MC) runs.

Loads all ensemble members from a pre-saved control-vector file in a single I/O call (controlvect.load(..., ensemble=True)), then applies scale2map() to the entire ensemble at once. This avoids the per-member I/O overhead of calling the standard forward() function nsamples times.

Two memory strategies are supported (selected via the mapper flag loadin_perturb_full_vertical):

  • Full vertical (default) — all levels are loaded for every member. Produces inout_datastore["outputs"][trid][ddi]["spec"] for each sample tracer ID.

  • Reduced vertical — only the first member uses the full vertical extent; subsequent members use only the surface level (index surface_level, default 0). This reduces peak memory at the cost of approximate upper-level perturbations.

Note

Physical control variables (type = 'physical') are not yet supported and will raise NotImplementedError.

Parameters:

  • transform (Plugin) – fromcontrol transform instance with dir_samples and file_samples attributes set by perturb_transform().

  • inout_datastore (dict) – mutable datastore.

  • controlvect – control vector plugin (x_ens, dx_ens loaded in-place).

  • obsvect – observation vector plugin (unused).

  • mapper (dict) – transform mapper; 'outputs' contains one entry per sample tracer ID (__sample#N).

  • di (datetime) – sub-simulation start date.

  • df (datetime) – sub-simulation end date.

  • mode (str) – 'fwd' or 'tl'.

  • runsubdir (str) – sub-simulation run directory.

  • workdir (str) – root working directory.

  • **kwargs – forwarded to tracer.read.

pycif.plugins.transforms.system.fromcontrol.perturb_transform.perturb_transform(self, nsamples, dir_samples, file_samples, transf_mapper)[source]#

Prepare the fromcontrol transform for ensemble (perturbation) mode.

Registers the ensemble control-vector file path on the transform so that forward_perturb() can load all members at once, and extends the datavect to include one tracer copy per sample (__sample#N naming convention). The reference (non-sampled) tracers are then removed from the datavect to avoid duplicate processing.

Parameters:

  • self (Plugin) – fromcontrol transform instance; receives perturb_xb = True, nsamples, dir_samples, and file_samples as new attributes.

  • nsamples (int) – total number of ensemble members.

  • dir_samples (str) – directory containing the ensemble control-vector file.

  • file_samples (str) – file name of the ensemble control vector.

  • transf_mapper (dict) – the transform mapper; its 'outputs' dict is scanned for __sample#N tracer IDs to set up.

pycif.plugins.transforms.system.fromcontrol.utils.dates.dateslice(tracer, di, df)[source]#

Gets the temporal chunk of the control vector corresponding to a simulation period.

Special cases:
  • one period spanning the whole inversion window:

    returns the whole table

  • initial conditions

Parameters:

  • tracer – the Tracer class with all its attributes

  • di – start/end dates of the simulations period

  • df – start/end dates of the simulations period

Returns:

list of date indexes and list of dates (including the end date)

pycif.plugins.transforms.system.fromcontrol.utils.scalemaps.scale2map(x, tracer, dates, dom, ensemble=False)[source]#

Unpack a control-vector slice to a spatial map at the model grid resolution.

Projects the 1-D (or 2-D ensemble) control-vector slice onto the full model domain according to the tracer’s horizontal resolution type:

  • hpixels — each element maps to one grid cell (direct reshape).

  • bands — each element fills a lat/lon band region.

  • ibands — each element fills a rectangular index-space band.

  • regions — each element fills a pre-defined geographic region (NaN outside any region).

  • global — a single element broadcasts over the entire domain.

Parameters:

  • x (np.ndarray) – control-vector slice of shape (ndates, vresoldim, nhresol) or (nsamples, ndates, vresoldim, nhresol) when ensemble is True.

  • tracer – tracer plugin instance with attributes hresol, vresoldim, levels, domain, and band/region definitions.

  • dates (array-like) – datetime array of length ndates used as the time coordinate of the output DataArray.

  • dom – domain plugin providing grid coordinates (zlon, zlat).

  • ensemble (bool) – if True, x carries a leading ensemble dimension and the output retains it.

Returns:

shape (time, lev, lat, lon) or (ens, time, lev, lat, lon) when ensemble is True.

Return type:

xarray.DataArray

Raises:

Exception – if tracer.hresol is not one of the recognised types.

pycif.plugins.transforms.system.fromcontrol.utils.scalemaps.map2scale(xmap, tracer, dom, region_scale_area=False, region_max_val=False)[source]#

Project a spatial sensitivity map onto the control-vector horizontal grid.

The adjoint of scale2map(): aggregates a model-space 4-D field (time, lev, lat, lon) to the control-vector horizontal resolution:

  • hpixels — reshape: each grid cell maps to one control element.

  • bands — sum over all cells in each lat/lon band.

  • ibands — sum over all cells in each index-space band.

  • regions — sum (or area-weighted mean, or max) over each region.

  • global — sum over the entire domain.

Parameters:

  • xmap (np.ndarray) – sensitivity field, shape (time, lev, lat, lon).

  • tracer – tracer plugin instance (same as passed to scale2map()).

  • dom – domain plugin providing grid coordinates.

  • region_scale_area (bool) – for regions resolution, weight each cell by its area and normalise by the total region area instead of summing.

  • region_max_val (bool) – for regions resolution, take the spatial maximum instead of the sum.

Returns:

aggregated slice of shape (ndates, vresoldim, nhresol).

Return type:

np.ndarray

Raises:

Exception – if the input does not have exactly 4 dimensions, or if tracer.hresol is not recognised.

pycif.plugins.transforms.system.fromcontrol.utils.scalemaps.vmap2vaggreg(data, tracer, dom, tracer_id)[source]#

Aggregate a full 3-D sensitivity field to the control-vector vertical resolution.

Three vertical resolutions are supported:

  • column — sum over all levels, returning a single vertical layer.

  • vpixels — keep each level individually (no aggregation); validates that the number of model levels matches tracer.vresoldim.

  • kbands — sum within each contiguous vertical band defined by tracer.kbands.

Parameters:

  • data (np.ndarray) – sensitivity field, shape (time, lev, lat, lon).

  • tracer – tracer plugin instance with vresol, vresoldim, nlev, and (for kbands) nvbands / kbands.

  • dom – domain plugin (unused; kept for API consistency).

  • tracer_id (tuple) – (component, parameter) for error messages.

Returns:

aggregated field of shape (time, vresoldim, lat, lon).

Return type:

np.ndarray

Raises:

Exception – if the input does not have 4 dimensions, if vpixels level count mismatches, or if vresol is not recognised.