pycif.plugins.models.chimere — API reference

pycif.plugins.models.chimere — API reference#

Configuration reference: chimere plugin

pycif.plugins.models.chimere.compile.compile(self)[source]#

Compile (or copy) the CHIMERE forward, TL, and adjoint executables.

Follows a two-strategy approach:

  1. Copy pre-compiled executables (default) — copies fwdchimere.e, tlchimere.e, and achimere.e from self.direxec into $workdir/model/. Skipped when force-recompile = True.

  2. Compile from sources — triggered when auto-recompile = True (or when the pre-compiled executables are not found). Clones the source tree from self.dir_sources (or self.direxec), patches the Makefiles for the configured compiler and library paths (NetCDF, GRIB, LDFLAGS), and runs make for each of the three build variants (forward, TL, adjoint) in PROD or DEBUG mode.

Parameters:

self (Plugin) – CHIMERE model plugin instance (carries workdir, direxec, dir_sources, compiler flags, and force-recompile / auto-recompile flags).

Raises:

RuntimeError – if auto-recompile is False and the executables cannot be found.

pycif.plugins.models.chimere.flushrun.flushrun(self, rundir, mode, transform_id, full_flush=True)[source]#

Cleaning the simulation directories to limit space usage

pycif.plugins.models.chimere.flushrun.flush_rundir(runsubdir, mode, final=False, full_flush=True)[source]#

Remove large temporary CHIMERE files from one sub-simulation directory.

Deletes the bulky input/output NetCDF files that are no longer needed once a sub-simulation period has completed (or once the full run is done).

Files always removed: AEMISSIONS, BEMISSIONS, BOUN_CONCS, INI_CONCS, METEO, chimere.nml, deposition, end, output files.

Additionally for adjoint runs: .txt files.

Additionally when final=True: aout, aini, aend files.

Additionally when full_flush=True: end.*.nc concentration restart files.

Parameters:

  • runsubdir (str) – path to the sub-simulation directory to clean.

  • mode (str) – execution mode ('fwd', 'tl', or 'adj').

  • final (bool) – if True, also remove aout, aini, aend files (called at end of full run, not between chained periods).

  • full_flush (bool) – if True, also remove end.*.nc restart files (disables period chaining restart capability).

pycif.plugins.models.chimere.ini_mapper.ini_mapper(model, general_mapper={}, backup_comps={}, transforms_order=[], ref_transform='', transform_name='', **kwargs)[source]#

Build the data-flow mapper for the CHIMERE chemistry-transport model.

Defines input and output tracer IDs and their associated domain and date-window metadata. CHIMERE requires the following input streams:

  • AEMISSIONS — anthropogenic emissions (nlevemis surface levels).

  • BEMISSIONS — biogenic emissions (surface level only).

  • Lateral boundary conditions — boundlat_north/south/east/west.

  • Top boundary conditions — boundtop.

  • Initial conditions — inicond (at datei only).

  • End concentrations — endconcs chained from the previous period.

  • Meteorology — wind, temperature, humidity, layer heights, etc.

Output streams are the simulated concentrations for each active species plus diagnostic meteorological fields (pressure, temperature, air mass, layer heights).

Also populates outputs2inputs linking each concentration output to the corresponding emission input to guide adjoint sensitivity routing.

Parameters:

  • model – CHIMERE model plugin instance.

  • general_mapper (dict) – unused.

  • backup_comps (dict) – unused.

  • transforms_order (list) – unused.

  • ref_transform (str) – unused.

  • transform_name (str) – unused.

  • **kwargs – unused.

Returns:

mapper with inputs, outputs, and outputs2inputs.

Return type:

dict

Raises:

PluginError – if required input components are missing from the datavect.

pycif.plugins.models.chimere.ini_periods.ini_periods(self, **kwargs)[source]#

Initialise sub-simulation periods and time-step grids for CHIMERE.

Partitions the full simulation window into sub-periods of length self.periods (typically 1 day) and determines the number of physical time steps per hour (nphour) for each sub-period.

When a METEO.nc file is present, nphour is read from the NetCDF nphourm variable, which may vary through the simulation window (CFL-adaptive time stepping). Otherwise it defaults to self.nphour_ref.

Sets the following attributes on self:

  • subsimu_dates — boundary dates of all sub-periods.

  • tstep_dates — dict mapping each sub-period start to its time-step array.

  • input_dates — same structure, used to schedule data reads.

  • tstep_all — flat array of all time steps.

  • nhour — dict of nphour per sub-period.

  • subtstep — dict of sub-time-step counts per sub-period.

  • iniobs — whether to dump observations for each sub-period.

  • reset_obs — whether to reset the observation accumulator.

  • runsimu — whether to actually run each sub-period (False for stop-or-more periods already computed).

  • chain — whether each sub-period has a successor.

Parameters:

  • self (Plugin) – CHIMERE model plugin instance (carries datei, datef, periods, nphour_ref, nho, nhours, workdir, and stopORmore).

  • **kwargs – unused; accepted for interface consistency.

Raises:

Exception – if sub-periods are not all of the same length.

pycif.plugins.models.chimere.perturb_model.perturb_model(self, nsamples, transf_mapper)[source]#

Extend the CHIMERE chemistry scheme to accommodate ensemble members.

Creates nsamples copies of each active species (acspecies), output species (outspecies), emitted species (emis_species), and biogenic species (bio_species) in the chemistry plugin, then removes the original un-suffixed species. The copies are named with a numeric suffix (e.g. CH4000, CH4001, …). Also records a perturbed_species mapping from sample name back to original name.

Note

Reactions in the chemical scheme are not yet supported with perturbations; an exception is raised if self.reactions is set.

Parameters:

  • self (Plugin) – CHIMERE model plugin instance.

  • nsamples (int) – number of ensemble members.

  • transf_mapper (dict) – transform mapper (unused; kept for API consistency with other model plugins).

Raises:

Exception – if the chemical scheme contains reactions.

pycif.plugins.models.chimere.run.run(self, runsubdir, mode, workdir, ddi, nbproc=1, do_simu=True, approx_transf=False, ref_fwd_dir='', overlap=False, **kwargs)[source]#

Run the CHIMERE model in forward mode

Parameters:

  • self – the model Plugin

  • runsubdir (str) – working directory for the current run

  • mode (str) – forward or backward

  • workdir (str) – pyCIF working directory

  • do_simu (bool) – re-run or not existing simulation

pycif.plugins.models.chimere.utils.check_inputs(self, runsubdir, mode)[source]#

Verify that all required CHIMERE input files exist in the run directory.

Checks for the presence of the mandatory input files before launching the CHIMERE executable. The list of required files depends on whether biogenic emissions are active (self.useemisb).

Mandatory files: METEO.nc, BOUN_CONCS.nc, AEMISSIONS.nc, INI_CONCS.0.nc, chimere.nml, chimere.e, and (if useemisb) BEMISSIONS.nc.

Parameters:

  • self (Plugin) – CHIMERE model plugin instance.

  • runsubdir (str) – sub-simulation run directory to check.

  • mode (str) – execution mode ('fwd', 'tl', or 'adj'); currently unused but kept for API consistency.

Raises:

Exception – if any required file is missing from runsubdir.

pycif.plugins.models.chimere.io.native2inputs.native2inputs(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Converts data at the model data resolution to model compatible input files.

Parameters:

  • self – the model Plugin

  • input_type (str) – one of ‘flux’

  • datastore – data to convert if input_type == ‘flux’,

  • datei – date interval of the sub-simulation

  • datef – date interval of the sub-simulation

  • mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’

  • runsubdir (str) – sub-directory for the current simulation

  • workdir (str) – the directory of the whole pyCIF simulation

Notes

  • CHIMERE expects hourly inputs;

pycif.plugins.models.chimere.io.native2inputs_adj.native2inputs_adj(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', check_transforms=False, **kwargs)[source]#

Converts data at the model data resolution to model compatible input files.

Parameters:

  • self – the model Plugin

  • input_type (str) – one of ‘flux’

  • datastore – data to convert if input_type == ‘flux’,

  • datei – date interval of the sub-simulation

  • datef – date interval of the sub-simulation

  • mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’

  • runsubdir (str) – sub-directory for the current simulation

  • workdir (str) – the directory of the whole pyCIF simulation

Notes

  • CHIMERE expects hourly inputs;

pycif.plugins.models.chimere.io.outputs2native.outputs2native(self, data2dump, input_type, di, df, runsubdir, mode='fwd', dump=True, onlyinit=False, check_transforms=False, **kwargs)[source]#

Reads outputs to pyCIF objects.

If the mode is ‘fwd’ or ‘tl’, only observation-like outputs are extracted. For the ‘adj’ mode, all outputs relative to model sensitivity are extracted.

Dumps to a NetCDF file with output concentrations if needed

pycif.plugins.models.chimere.io.outputs2native_adj.outputs2native_adj(self, data2dump, input_type, di, df, runsubdir, mode='fwd', dump=True, onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Reads outputs to pyCIF objects.

If the mode is ‘fwd’ or ‘tl’, only observation-like outputs are extracted. For the ‘adj’ mode, all outputs relative to model sensitivity are extracted.

Dumps to a NetCDF file with output concentrations if needed

pycif.plugins.models.chimere.io.inputs.make_auxiliary.make_auxiliary(self, ddi, runsubdir, onlyinit=False, do_simu=True, mode='fwd', **kwargs)[source]#

Set up the CHIMERE executable and namelist for one sub-simulation period.

Symlinks the correct CHIMERE binary (fwdchimere.e, tlchimere.e, or achimere.e) into runsubdir as chimere.e, then calls make_nml() to write the Fortran namelist chimere.nml.

If onlyinit=True the function returns immediately without doing anything (the adjoint init pass uses previously linked forward outputs).

Parameters:

  • self – CHIMERE model plugin instance with workdir set.

  • ddi (datetime) – start date of the sub-simulation period.

  • runsubdir (str) – path to the period run directory.

  • onlyinit (bool) – if True, skip setup (used during adjoint initialisation).

  • do_simu (bool) – if False, skip setup.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • **kwargs – unused.

Raises:

Exception – if mode is not one of the recognised values.

pycif.plugins.models.chimere.io.inputs.make_boundcond.make_boundcond(self, datastore, runsubdir, ddi, mode, input_type)[source]#

Generates boundary conditions files for CHIMERE

Parameters:

  • self

  • datastore (dict)

  • runsubdir (str)

  • sdc (str)

  • hour_dates

  • mode

  • input_type

Returns:

pycif.plugins.models.chimere.io.inputs.make_endconcs.make_endconcs(self, datastore, runsubdir, mode, datei, datef, onlyinit, check_transforms=False)[source]#

Link concentration restart files into the CHIMERE period run directory.

Handles four cases depending on mode and onlyinit:

  • Forward / TL, not onlyinit — links the end-concentrations produced by the previous period as INI_CONCS.0.nc for the current period.

  • Adjoint, onlyinit — same as above (links forward end-concs as initial state for the backward adjoint).

  • Adjoint, not onlyinit — links the adjoint initial conditions (from the forward reference run stored in self.adj_refdir) as aini.nc.

Parameters:

  • self – CHIMERE model plugin instance (carries adj_refdir for adjoint runs).

  • datastore (dict) – mapping of tracer IDs to their data-store entries, each containing fileorig paths.

  • runsubdir (str) – path to the period run directory.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • datei (datetime) – period start date.

  • datef (datetime) – period end date.

  • onlyinit (bool) – True during the adjoint initialisation pass.

  • check_transforms (bool) – unused, kept for API compatibility.

pycif.plugins.models.chimere.io.inputs.make_fluxes.make_fluxes(self, datastore, runsubdir, datei, mode)[source]#

Make AEMISSIONS.nc and BEMISSIONS.nc files for CHIMERE. Use chemical scheme to check which species is needed and either take it from the datastore (i.e. when defined in the control vector), or take it from prescribed emissions

Parameters:

  • self (pycif.utils.classes.fluxes.Flux) – Flux plugin with all attributes

  • datastore (dict) – information on flux species

  • runsubdir (str) – directory to the current run

  • nho (int) – number of hours in the run

  • mode (str) – running mode: ‘fwd’, ‘tl’ or ‘adj’

pycif.plugins.models.chimere.io.inputs.make_inicond.make_inicond(self, datastore, runsubdir, mode, datei, datef, ini_type='inicond')[source]#

Write or symlink the CHIMERE initial concentration file INI_CONCS.0.nc.

For the first sub-simulation period only (datei == self.datei). For each active species:

  • If no data was modified in the CIF datastore, symlinks the original file from disk.

  • If concentrations were modified (e.g. by a prior transform), copies the original file and overwrites the relevant species variable.

  • For tangent-linear mode, also creates INI_CONCS.0.increment.nc initialised to zero perturbations.

  • Handles ensemble/perturbed species by reading from the reference species and writing to the perturbed copy.

After all species are written, calls replace_dates() to ensure the NetCDF time axis matches the CHIMERE convention.

Parameters:

  • self – CHIMERE model plugin instance (carries datei, nho, adj_refdir, chemistry.acspecies, and inicond).

  • datastore (dict) – tracer-ID-keyed mapping of CIF data-store entries.

  • runsubdir (str) – path to the period run directory.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • datei (datetime) – period start date.

  • datef (datetime) – period end date.

  • ini_type (str) – datastore key prefix; 'inicond' for normal use or 'restart_inicond' in EnSRF restart mode.

pycif.plugins.models.chimere.io.inputs.make_meteo.make_meteo(self, datastore, runsubdir, mode, datei)[source]#

Make METEO.nc files for CHIMERE.

Parameters:

  • self (pycif.utils.classes.) – plugin with all attributes

  • datastore (dict) – information on meteo variables

  • runsubdir (str) – directory to the current run

  • nho (int) – number of hours in the run

  • mode (str) – running mode: ‘fwd’, ‘tl’ or ‘adj’

pycif.plugins.models.chimere.io.inputs.make_obs.make_obs(self, ddi, dict_datastores, runsubdir, mode, list_tracer, input_type, do_simu=True)[source]#

Write the CHIMERE observation text file obs.txt for one sub-period.

Accumulates observation metadata (grid indices, time steps, species, dtstep) into the CHIMERE-formatted obs.txt file. For adjoint runs the departure vector (adj_out) is appended as an extra column.

Handles:

  • Auxiliary outputs (pressure, temperature, …): only stores count metadata in self.auxiliary_indexes; returns early without writing.

  • do_simu=False: updates obs-count bookkeeping without writing a file (used when the forward output was already cached).

  • Multi-timestep observations (dtstep > 1) are expanded row-by-row so each line covers exactly one CHIMERE sub-time-step.

  • Coordinate and level indices are converted from Python (0-based) to Fortran (1-based) before writing.

Parameters:

  • self – CHIMERE model plugin instance (carries nhour, subtstep, iniobs, reset_obs, chemistry.outspecies, and output_components).

  • ddi (datetime) – sub-simulation period start.

  • dict_datastores (dict) – species-keyed CIF data-store mapping.

  • runsubdir (str) – path to the period run directory.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • list_tracer (list) – species names to write in this call.

  • input_type (str) – output component name ('concs', 'pressure', etc.).

  • do_simu (bool) – if False, skip writing and only update bookkeeping.

pycif.plugins.models.chimere.io.inputs.params.make_nml(self, runsubdir, sdc, mode, ddi)[source]#

Write the Fortran namelist chimere.nml for one sub-simulation period.

Generates the &args namelist that controls the CHIMERE executable. All domain dimensions, chemical scheme paths, I/O file paths, physics switches, and run mode flags are written from the plugin’s attributes.

The output file is {runsubdir}/chimere.nml.

Mode flags written: fwd, tl, ad (only one is set to 1).

Key physics switches (sourced from plugin attributes): usechemistry, usedepos, useemissions, usetransmix, usewetdepos, useabsclipconc, optemisb.

Parameters:

  • self – CHIMERE model plugin instance with all required domain, chemistry, and configuration attributes set.

  • runsubdir (str) – path to the period run directory.

  • sdc (str) – period start date formatted as '%Y%m%d%H'.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • ddi (datetime) – period start datetime (used to look up self.chain for the adjoint initial-condition flag iopainit).

pycif.plugins.models.chimere.io.inputs.utils.replace_dates(file_emisout, list_dates, ignore_dates)[source]#

Check (or ignore) that a CHIMERE NetCDF file’s Times axis matches expected dates.

CHIMERE expects every input NetCDF to carry a Times variable encoded as %Y-%m-%d_%H:00:00 strings. If the dates in the file differ from list_dates:

  • ignore_dates=True: emits a warning and continues (useful when reusing files from a different time window).

  • ignore_dates=False: raises an Exception describing the mismatch and advising the user to set ignore_input_dates: true in the YAML.

Parameters:

  • file_emisout (str) – path to the NetCDF file to check.

  • list_dates (list[datetime]) – expected sequence of dates.

  • ignore_dates (bool) – if True, suppress the mismatch error.

Raises:

Exception – if dates mismatch and ignore_dates is False.

pycif.plugins.models.chimere.io.outputs.fetch_end.fetch_end(self, data2dump, runsubdir, mode, ddi, ddf, check_transforms=False, onlyinit=False)[source]#

Register end-concentration output paths (and optionally read data) after a CHIMERE run.

For adjoint mode: records the aend.*.nc file path in data2dump, and when check_transforms is active, reads the adjoint sensitivity fields (_ad_o and _ad) from the distributed tile layout back into the data store.

For forward/TL mode: records the end.*.nc file path, and when check_transforms is active in TL mode, reads the TL increment (_o_tl and _tl) back into the data store.

Parameters:

  • self – CHIMERE model plugin instance (carries nho, domain, nzdoms, nmdoms).

  • data2dump (dict) – tracer-ID-keyed data-store entries to update.

  • runsubdir (str) – path to the period run directory.

  • mode (str) – 'fwd', 'tl', or 'adj'.

  • ddi (datetime) – period start date.

  • ddf (datetime) – period end date (unused, kept for API consistency).

  • check_transforms (bool) – if True, also read output fields back into data2dump (used by the transform-checker).

  • onlyinit (bool) – if True, skip reading fields (initialisation pass).

Returns:

updated data2dump with fileorig paths (and optionally concentration / sensitivity arrays) set.

Return type:

dict

pycif.plugins.models.chimere.io.outputs.fetch_end.make_end(self, file_end, ddi, ref_fwd_dir)[source]#

Create a zeroed TL end-concentration file for tangent-linear initialisation.

Reads the reference forward end file (chain/end.*.nc in ref_fwd_dir), appends zero-initialised TL increment variables for all active species plus the fixed-species set (M, O2, N2, H2O), and writes the result to file_end.

Both the forward (<spec>) and output-level (<spec>_o_tl) TL variables are initialised to zero.

Parameters:

  • self – CHIMERE model plugin instance (carries nho, domain, chemistry.acspecies).

  • file_end (str) – output path for the initialised TL file.

  • ddi (datetime) – period start date (used to build the filename).

  • ref_fwd_dir (str) – directory containing the reference forward chain.

Raises:

Exception – if the reference forward end file does not exist.

pycif.plugins.models.chimere.io.outputs.fetch_end.make_aend(self, file_end, ddi)[source]#

Create a zeroed adjoint end-sensitivity file for adjoint initialisation.

Writes {file_end} as a NetCDF3 CLASSIC file containing zeroed sensitivity fields for all active and fixed species (M, O2, N2, H2O). Both the full-domain (<spec>_ad) and output- level (<spec>_ad_o) arrays are included, accounting for the MPI tile buffer padding (6 * nzdoms / 6 * nmdoms cells).

Parameters:

  • self – CHIMERE model plugin instance (carries domain, chemistry.acspecies, nzdoms, nmdoms, nivout).

  • file_end (str) – output path for the initialised adjoint file.

  • ddi (datetime) – period start date (written into the Times variable).

pycif.plugins.models.chimere.io.outputs.make_aout.make_aout(self, runsubdir, ddi)[source]#

Create zeroed adjoint output files expected by the CHIMERE adjoint executable.

Writes four NetCDF3 CLASSIC files into runsubdir:

  • aout.aemis.nc — adjoint sensitivities w.r.t. anthropogenic emissions; shape (nhours+1, nlevemis, nlat, nlon) per species.

  • aout.bemis.nc — adjoint sensitivities w.r.t. biogenic emissions; shape (nhours+1, nlevemis_bio, nlat, nlon) per species.

  • aout.ini.nc — adjoint sensitivities w.r.t. initial concentrations; shape (nivout, nlat, nlon) per species.

  • aout.bc.nc — adjoint sensitivities w.r.t. boundary conditions; contains top_conc and lat_conc tensors.

All arrays are initialised to zero; the CHIMERE adjoint executable accumulates sensitivities into them during the backward run.

Parameters:

  • self – CHIMERE model plugin instance (carries domain, chemistry.acspecies, chemistry.emis_species, chemistry.bio_species, nhours, nlevemis, nlevemis_bio, nivout).

  • runsubdir (str) – path to the period run directory.

  • ddi (datetime) – period start date (unused; kept for API symmetry with other io functions).

pycif.plugins.models.chimere.io.outputs.read_sim.read_sim(self, data2load, runsubdir, mode, ddi, ddf)[source]#

Read the mod.txt file as simulated by CHIMERE

pycif.plugins.models.chimere.io.outputs.read_sim.read_obs(runsubdir, mode='fwd')[source]#

Read obs.txt file

pycif.plugins.models.chimere.io.outputs.read_sim.write_sim(data, runsubdir)[source]#

Write mock mod.txt file

pycif.plugins.models.chimere.io.outputs.read_sim.write_obs(data, runsubdir)[source]#

Write mock obs.txt file

pycif.plugins.models.chimere.io.outputs.read_sim.readend(ficin)[source]#

Read end.nc file

pycif.plugins.models.chimere.io.outputs.read_sim.readmeteo(ficin, list_var)[source]#

Read variables in METEO.nc file