pycif.plugins.models.dalecbethy — API reference

pycif.plugins.models.dalecbethy — API reference#

Configuration reference: dalecbethy plugin

pycif.plugins.models.dalecbethy.ini_mapper.ini_mapper(model, general_mapper={}, backup_comps={}, transforms_order=[], ref_transform='', transform_name='', all_transforms=None, **kwargs)[source]#

Build the data-flow mapper for the D&B bottom-up CO2 flux model.

Declares:

  • Inputs:

    • D&B’s own control vector: the prior parameters read from input/{core,sif,lvod,slope}-params.csv (src/prior.f90), following the same ("<model>_param", <component>) convention as satwetch4’s ("satwetch4_model_param", "k"/"q10") inputs.

    • The time-invariant land-surface/PFT fields read from input/staticforcing.nc (src/ncread_forcing.f90:: ncread_static_forcing), as ("dalecbethy_static", <component>).

    • The hourly meteorological forcing read from input/dynforcing.nc (src/ncread_forcing.f90:: ncread_dynamic_forcing), as ("meteo", <component>), following the same convention as chimere’s/satwetch4’s ("meteo", ...) inputs.

  • Outputs – the NEE (net ecosystem exchange) flux computed by D&B (src/obsop.f90/src/timeloop.f90, dumped to diagout/dalec-bethy_{hourly,daily}-output_*.nc, variable nee, see src/ncwrite_output.f90), following the same ("flux", <component>) convention as satwetch4’s ("flux", "CH4_wetlands") output.

Warning

TODO for a D&B developer:

  1. Both the dynamic (meteo) and static forcing declared here are wired through native2inputs()/ make_meteo()/ make_static(), which (re)write input/dynforcing.nc/input/staticforcing.nc from the datastore instead of linking the original files (see make_auxiliary(), which no longer links either of them).

  2. The four ("dalecbethy_param", <kind>) inputs are scalar-per-PFT values (one row per varname/PFT pair, see the parameters CSV files), not gridded fields, so the gridded_netcdf transport used by satwetch4 for model_param_k/model_param_q10 likely needs dedicated handling in native2inputs() rather than applying as-is.

Parameters:

  • model – dalecbethy plugin instance with all date arrays set.

  • general_mapper (dict) – unused.

  • backup_comps (dict) – unused.

  • transforms_order (list) – unused.

  • ref_transform (str) – unused.

  • transform_name (str) – unused.

  • all_transforms – unused.

  • **kwargs – unused.

Returns:

mapper with inputs and outputs.

Return type:

dict

pycif.plugins.models.dalecbethy.ini_periods.ini_periods(self, **kwargs)[source]#

Define D&B’s temporal splitting variables.

Sets subsimu_dates, tstep_dates and input_dates following the template/satwetch4 convention.

Warning

TODO for a D&B developer: D&B’s simulation window (yyyymmdd_start/yyyymmdd_end) and internal hourly time step are compiled into the executable via src/dimensions.f90 (itself generated from util/templates/dimensions.f90.template by util/model_setup.py at make time, see compile.py). D&B therefore cannot, as of today, be split into several independently-run pyCIF sub-simulations without triggering a recompilation for each one (unlike e.g. CHIMERE, which reads its simulation window from a runtime namelist).

Until this is resolved (either by adding a runtime override to D&B, or by re-triggering compile() for every sub-period), this function defines a single sub-simulation spanning the full [datei, datef] pyCIF window, and internal time steps at hourly resolution (D&B’s native resolution, see src/dimensions.f90: nhour = nday*24), regardless of output_resolution.

Parameters:

  • self – the dalecbethy model Plugin.

  • **kwargs – unused.

pycif.plugins.models.dalecbethy.run.run(self, runsubdir, mode, workdir, ddi, nbproc=1, do_simu=True, approx_transf=False, ref_fwd_dir='', overlap=False, datastore=None, **kwargs)[source]#

Compile (if needed) and run the D&B executable for one sub-period.

D&B is a self-contained Fortran executable (not a Python model like satwetch4): this function launches it as a subprocess in runsubdir, where make_auxiliary() and native2inputs() are expected to have already staged everything the executable needs (input/*.nc, input/*-params.csv, opt.nml, obs.nml), following the fixed, hard-wired relative paths used throughout the D&B sources (see e.g. src/prior.f90, src/ncread_forcing.f90: input/..., mode==adj is not implemented yet.

This function also triggers compile() itself, on the first call only (i.e. once self.workdir/model/runmodel.x exists, later calls skip straight to running it). It is deliberately not triggered any earlier – e.g. by the generic, input-agnostic run_model transform init (see the __init__.py note) – because D&B’s build needs runsubdir/input/{static,dyn}forcing.nc to already exist (see compile()); by the time run is first called, pyCIF has finished initialising the model (domain, mapper, periods) and staging its input data (via native2inputs()), so that precondition holds.

Parameters:

  • self – the dalecbethy model Plugin.

  • runsubdir (str) – working directory for the current run; the D&B executable and its input/ directory must be staged here.

  • mode (str) – ‘fwd’, ‘tl’ or ‘adj’.

  • workdir (str) – pyCIF working directory.

  • ddi (datetime.datetime) – start date of the current sub-simulation.

  • do_simu (bool) – re-run or not an existing simulation.

Warning

TODO / open questions for a D&B developer:

  • D&B ships a hand-coded adjoint (src/cost_bw.f90, runassi.x) but it is wired to D&B’s own L-BFGS-B minimiser (mini/), driving D&B’s internal control vector (src/prior.f90) against D&B’s own observation operator (src/obsop.f90), not to a per-timestep tangent-linear/adjoint call pyCIF could drive itself (unlike e.g. CHIMERE’s tlchimere.e/achimere.e). Whether/how a pyCIF-driven mode='tl'/'adj' run should be implemented here (e.g. by building and calling runassi.x or a new dedicated driver) is left as a follow-up.

  • D&B currently only supports full-domain runs at hourly internal time steps between yyyymmdd_start/yyyymmdd_end (src/dimensions.f90); how that maps onto pyCIF’s sub-simulation windows (ddi/ddf) still needs to be decided, in particular since those dates are presently compiled into the executable rather than passed at runtime.

pycif.plugins.models.dalecbethy.utils.compile(self, runsubdir)[source]#

Compile (or copy) the D&B forward executable (runmodel.x).

Follows the same two-strategy approach as the chimere plugin:

  1. Copy a pre-built executable (default) – copies runmodel.x from self.direxec into $workdir/model/. Skipped when force-recompile = True.

  2. Compile from sources – triggered when auto-recompile = True (or when no pre-built executable is found). Clones the source tree from self.dir_sources (which should point to model_sources/dalecbethy, i.e. the unmodified mirror of /home/chimereges/aberchet/DB/model/sites), links the compiler configuration file (mk.compile -> config/mk.compile-<compiler>, see the D&B README), links runsubdir’s already-staged input/{static,dyn}forcing.nc into the source tree, and runs make runmodel.x DOMAIN=<domain_name>.

Note

D&B’s runmodel.x target does not merely link object files: it transitively depends on src/dimensions.f90, which the Makefile regenerates (via util/model_setup.py) from the actual $(INDIR)/{static,dyn}forcing.nc files, i.e. input/staticforcing.nc / input/dynforcing.nc (see the Makefile rule for $(SRCDIR)/dimensions.f90 and the $(FORCE_STATIC)/ $(FORCE_DYN) targets). Those are exactly the files native2inputs() already wrote to runsubdir/input/ before run() gets here, so this function links them straight into sources/input/ under the same names, which pre-satisfies the $(FORCE_STATIC)/ $(FORCE_DYN) targets and short-circuits the Makefile’s own forcing/<domain_name>_{static,dyn}forcing*.nc lookup. This is also why run() – not the generic, input-agnostic run_model transform init – is what triggers this function: runsubdir/input must already hold the real, resolved forcing files for the compile to succeed.

The prior parameter CSVs (parameters/<domain_name>-*-params.csv) do not need the same treatment: they are small enough to be part of the tracked model_sources/dalecbethy/parameters mirror already, so the Makefile’s own $(INDIR)/%params.csv rule resolves them directly from the copied source tree.

Warning

TODO / open question for a D&B developer: the tangent-linear/adjoint executable is only built by the xassi/libs targets (libadstack-$(FC).a, libmini-$(FC).a, cost_bw.f90); this is not wired up here yet since D&B’s own TL/adjoint currently only feeds its internal L-BFGS-B minimiser (make xassi), not a pyCIF-driven tl/adj run.

Parameters:

  • self (Plugin) – dalecbethy model plugin instance (carries workdir, direxec, dir_sources, domain_name, force-recompile / auto-recompile flags).

  • runsubdir (str) – sub-directory of the simulation that triggered this (lazy, first-call-only) compilation; its input/ directory is expected to already hold staticforcing.nc/dynforcing.nc, as staged by native2inputs().

Raises:

CifRuntimeError – if auto-recompile is False and no executable can be found in direxec.

pycif.plugins.models.dalecbethy.io.native2inputs.native2inputs(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Converts data at the model data resolution to D&B-compatible input files.

Mirrors pycif.plugins.models.chimere.io.native2inputs.native2inputs: dispatches on input_type to a dedicated make_<type> function. "meteo" (D&B’s input/dynforcing.nc) and "dalecbethy_static" (input/staticforcing.nc) are wired so far, per required_inputs in the package __init__.py (see the ini_mapper.py TODO for the still-unwired "dalecbethy_param" component type).

Parameters:

  • self – the dalecbethy model Plugin.

  • datastore – data to convert.

  • input_type (str) – the component type to convert; "meteo" and "dalecbethy_static" are handled.

  • datei – date interval of the sub-simulation.

  • datef – date interval of the sub-simulation.

  • runsubdir (str) – sub-directory for the current simulation.

  • mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’.

  • onlyinit (bool) – unused, kept for interface compatibility.

  • do_simu (bool) – if False, the simulation does not need to be run.

  • check_transforms (bool) – unused, kept for interface compatibility.

  • **kwargs – unused, kept for interface compatibility.

pycif.plugins.models.dalecbethy.io.native2inputs_adj.native2inputs_adj(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', check_transforms=False, **kwargs)[source]#

Read adjoint sensitivities and format them to pyCIF data format.

Warning

Not implemented: D&B’s adjoint (src/cost_bw.f90) is currently only wired to D&B’s own internal minimiser (make xassi, see the TODO in run.py), not to a pyCIF-driven adjoint run. There is therefore nothing to convert here yet.

Parameters:

  • self – the dalecbethy model Plugin.

  • input_type (str) – component name.

  • datastore – data to convert.

  • datei – date interval of the sub-simulation.

  • datef – date interval of the sub-simulation.

  • mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’.

  • runsubdir (str) – sub-directory for the current simulation.

pycif.plugins.models.dalecbethy.io.outputs2native.outputs2native(self, data2dump, input_type, di, df, runsubdir, mode='fwd', onlyinit=False, check_transforms=False, **kwargs)[source]#

Read D&B NetCDF diagnostics back into pyCIF objects.

D&B writes its diagnostics to diagout/dalec-bethy_{hourly,daily}-output_<yyyymmdd_start>-<yyyymmdd_end>.nc (see src/ncwrite_output.f90), on the nsp sample-point dimension (grid-cell x active-PFT combinations, see src/dimensions.f90). The stream (hourly/daily) to read is controlled by self.output_resolution.

The output file also carries, on that same nsp dimension, the gidx (1-based grid-cell index, D&B’s map_sample2grid) and pft_fraction (D&B’s fracv) static fields. These are used here to recombine the PFT-level sample points of each grid cell into a single per-cell value (a pft_fraction-weighted average), so the result lines up with the pyCIF domain built by pycif.plugins.domains.dalecbethy.read_domain (domain.active, on the same grid-cell indexing as gidx).

Parameters:

  • self – the dalecbethy model Plugin.

  • data2dump (dict) – output data structure to fill for every component/tracer declared in the mapper.

  • input_type (str) – the type of model outputs to be processed (redundant with the components of data2dump).

  • di (datetime.datetime) – start/end date of the present sub-simulation.

  • df (datetime.datetime) – start/end date of the present sub-simulation.

  • runsubdir (str) – path to the present sub-simulation work directory.

  • mode (str) – running mode; one of “fwd”, “tl” and “adj”.

Returns:

a dictionary with structure the components/tracers to be extracted, with each component’s per-grid-cell xarray.DataArray (PFT sample points recombined onto self.domain’s grid cells).

Return type:

dict

pycif.plugins.models.dalecbethy.io.outputs2native_adj.outputs2native_adj(self, data2dump, input_type, di, df, runsubdir, mode='fwd', onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Save adjoint-forcing information for a later adjoint D&B call.

Warning

Not implemented: see the run.py TODO – D&B’s adjoint is not yet wired to be driven step-by-step from pyCIF.

Parameters:

  • self – the dalecbethy model Plugin.

  • data2dump (dict) – concentration/flux data for each component.

  • input_type (str) – the type of model outputs to be processed.

  • di (datetime.datetime) – start/end date of the present sub-simulation.

  • df (datetime.datetime) – start/end date of the present sub-simulation.

  • runsubdir (str) – path to the present sub-simulation work directory.

  • mode (str) – running mode; one of “fwd”, “tl” and “adj”.

pycif.plugins.models.dalecbethy.io.inputs.make_auxiliary.make_auxiliary(self, ddi, runsubdir, do_simu=True, mode='fwd', **kwargs)[source]#

Stage every file D&B’s runmodel.x expects, in runsubdir.

D&B hard-wires relative input paths in its Fortran sources (e.g. src/prior.f90: input/core-params.csv; src/ncread_forcing.f90: input/staticforcing.nc, input/dynforcing.nc; src/obs.f90: obs.nml), so this function must reproduce, inside runsubdir, the same layout as the reference D&B checkout (see the Jobs/Makefile “model_input” target for the equivalent shell-level logic):

  • neither runsubdir/input/staticforcing.nc nor runsubdir/input/dynforcing.nc are linked here: both are (re)written per sub-simulation, from the ("dalecbethy_static", <component>)/("meteo", <component>) data-store entries, by native2inputs()/ make_static()/make_meteo() (see pycif.plugins.models.dalecbethy.ini_mapper.ini_mapper());

  • link the four prior-parameter CSV files for self.domain_name (from self.parameters_dir) to runsubdir/input/{core,sif,lvod,slope}-params.csv;

  • write runsubdir/obs.nml (see _OBS_NML_DEFAULTS);

  • create the (empty) runsubdir/diagout directory D&B writes its NetCDF diagnostics to.

The compiled executable itself is staged into runsubdir by run(), not here – run.run is also what lazily triggers compile() on first use (see the package __init__.py note on why compilation cannot happen any earlier).

Parameters:

  • self – the dalecbethy model plugin.

  • ddi (datetime.datetime) – start date of the present simulation period.

  • runsubdir (str) – path to the current sub-simulation work directory.

  • do_simu (bool) – if False, the simulation does not need to be run.

  • mode (str) – the running mode (‘fwd’, ‘tl’, ‘adj’).

pycif.plugins.models.dalecbethy.io.inputs.make_forcing.make_meteo(self, datastore, runsubdir, mode, datei, datef)[source]#

Build D&B’s input/dynforcing.nc from the meteo data-store.

See _make_forcing() for the shared logic (also used by make_static()): for each of D&B’s mandatory hourly meteo components (see pycif.plugins.models.dalecbethy.ini_mapper.meteo_components), either the raw forcing file is linked as-is, or the component is (re)written via self.meteo.write. The mandatory yyyymmddhh calendar variable – not itself a per-component mapper input – is then copied as-is from the original raw file, for the hours covered by [datei, datef].

Parameters:

  • self – the dalecbethy model Plugin.

  • datastore (dict) – the pyCIF data-store, keyed by ("meteo", <component>).

  • runsubdir (str) – sub-directory for the current simulation.

  • mode (str) – running mode (‘fwd’, ‘tl’, ‘adj’); only ‘fwd’ is supported so far (D&B has no adjoint/tangent-linear wiring for meteo yet).

  • datei – date interval of the sub-simulation.

  • datef – date interval of the sub-simulation.

Raises:

NotImplementedError – if mode != "fwd".

pycif.plugins.models.dalecbethy.io.inputs.make_forcing.make_static(self, datastore, runsubdir, mode, datei, datef)[source]#

Build D&B’s input/staticforcing.nc from the dalecbethy_static data-store.

See _make_forcing() for the shared logic (also used by make_meteo()): for each of D&B’s time-invariant land-surface/PFT components (see pycif.plugins.models.dalecbethy.ini_mapper.static_components), either the raw forcing file is linked as-is, or the component is (re)written via self.static.write. Unlike dynforcing.nc, there is no yyyymmddhh calendar variable to copy.

Parameters:

  • self – the dalecbethy model Plugin.

  • datastore (dict) – the pyCIF data-store, keyed by ("dalecbethy_static", <component>).

  • runsubdir (str) – sub-directory for the current simulation.

  • mode (str) – running mode (‘fwd’, ‘tl’, ‘adj’); only ‘fwd’ is supported so far (D&B has no adjoint/tangent-linear wiring for static forcing yet).

  • datei – date interval of the sub-simulation.

  • datef – date interval of the sub-simulation.

Raises:

NotImplementedError – if mode != "fwd".

pycif.plugins.models.dalecbethy.io.inputs.utils.find_unique_match(directory, pattern)[source]#

Find the single file matching pattern in directory.

Parameters:

  • directory (str) – directory to search in.

  • pattern (str) – a glob pattern.

Returns:

the single matching file path.

Return type:

str

Raises:

FileNotFoundError – if zero, or more than one, file matches.