pycif.plugins.models.template — API reference

`pycif.plugins.models.template` — API reference#

Configuration reference: template plugin

pycif.plugins.models.template.compile.compile(self)[source]#

The compile function initializes all model information and executables prior to any run. Files must be copied in $workdir/model.

This includes:

copying executable if exist

Warning

It is recommended to copy executable files to make sure than later simulations in the present pyCIF computation use the same executable. Indeed, it can happen that one runs very long inversions in the background and carries on developments, forgetting about the background inversions, thus potentially breaking the background inversions, or worse, changing the result without error…

copy sources and compile if no executable is around, or if explicitly required to re-compile.

Note

As much as possible, the model should be compiled within pyCIF to guarantee a traceability of the options used for compiling and also dealing with platform specificities through the platform Plugin (see details here)

copy extra configuration files, e.g., templates for namelists, etc.

pycif.plugins.models.template.ini_mapper.ini_mapper(model, general_mapper={}, backup_comps={}, transforms_order=[], ref_transform='', transform_name='', all_transforms=None, **kwargs)[source]#

Build the data-flow mapper for the template model.

Constructs a minimal mapper controlled by input_arguments flags:

empty_mapper=True — returns an empty mapper (no inputs/outputs).
only_outputs=True — returns a mapper with concentration outputs only.
Otherwise — returns a mapper with both flux inputs and concentration outputs.

Parameters:

model – template plugin instance.
general_mapper (dict) – unused.
backup_comps (dict) – unused.
transforms_order (list) – unused.
ref_transform (str) – unused.
transform_name (str) – unused.
all_transforms – unused.
**kwargs – unused.

Returns:

mapper with inputs, outputs, and optionally outputs2inputs.

Return type:

dict

pycif.plugins.models.template.ini_periods.ini_periods(self, **kwargs)[source]#

The function ini_periods is optional but very recommended. It is used to define the temporal variables subsimu_dates, input_dates, tstep_dates and tstep_all. The function is automatically called at the initialization of the model class if available. If not available, the temporal variables should be defined manually in the ini_data function (not recommended).

ini_periods is a class method that applies to the model plugin itself. Therefore, the only expected argument is self.

def ini_periods(self, **kwargs):

    self.subsimu_dates = XXXX
    self.tstep_dates = XXXXX
    self.input_dates = XXXXX
    self.tstep_all = XXXXX

pycif.plugins.models.template.run.run(self, runsubdir, mode, workdir, ddi, nbproc=1, do_simu=True, approx_transf=False, ref_fwd_dir='', overlap=False, **kwargs)[source]#

Run the model in forward, tangent-linear or adjoint mode. This includes:

executing the model external executable

updating adj_refdir

moving files needed for chained simulations to “{}/../”.format(runsubdir)

Note

For model for which the adjoint is not coded, make sure to return a clear error if the run function is called in adj mode and with do_simu = True

Parameters:

self – the model Plugin
runsubdir (str) – working directory for the current run
mode (str) – forward or backward
workdir (str) – pyCIF working directory
do_simu (bool) – re-run or not existing simulation

pycif.plugins.models.template.io.native2inputs.native2inputs(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Converts data at the model data resolution stored in datastore to model compatible input files.

Native2inputs will be called for every couple component/tracer as defined in the mapper

Parameters:

self – the model Plugin
input_type (str) – the component name to be treated; please note that this information is redundant with the keys in datastore
datastore – data to convert
datei – date interval of the sub-simulation
datef – date interval of the sub-simulation
mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’
runsubdir (str) – sub-directory for the current simulation
workdir (str) – the directory of the whole pyCIF simulation

Note

The format of datastore is a mixture of the model mapper and of the data format as defined here

For each component/tracer, the data itself is stored in the key data, and all the other keys come from the mapper, in case they are useful to dump inputs at the correct format

Note

If the input data was fully consistent with what the model expects, the data itself is not read by pyCIF. Instead, it is possible to directly link files defined by the key input_files (and defined in the fetch function of the corresponding flux plugin).

pycif.plugins.models.template.io.native2inputs_adj.native2inputs_adj(self, datastore, input_type, datei, datef, runsubdir, mode='fwd', check_transforms=False, **kwargs)[source]#

Read adjoint sensitivity and format them to pyCIF data format.

Warning

This function is used only when the adjoint of the model is available.

Parameters:

self – the model Plugin
input_type (str) – one of ‘flux’
datastore – data to convert if input_type == ‘flux’,
datei – date interval of the sub-simulation
datef – date interval of the sub-simulation
mode (str) – running mode: one of ‘fwd’, ‘adj’ and ‘tl’
runsubdir (str) – sub-directory for the current simulation
workdir (str) – the directory of the whole pyCIF simulation

pycif.plugins.models.template.io.outputs2native.outputs2native(self, data2dump, input_type, di, df, runsubdir, mode='fwd', onlyinit=False, check_transforms=False, **kwargs)[source]#

Reads outputs to pyCIF objects.

Parameters:

self – the model itself
data2dump (dict) – a dictionary with output data structure to be filled with correct data for every component/tracer
input_type (str) – the type of model outputs to be processed; this information is redundant with the components of the data2dump dictionary
di (datetime.datetime) – starting date of the present sub-simulation
df (datetime.datetime) – ending date of the present sub-simulation
runsubdir (str) – path to the present sub-simulation work directory
mode (str) – running mode; one of “fwd”, “tl” and “adj”
onlyinit (bool) – if True, means that the function is called during the initialization process of the observation vector
do_simu (bool) – if False, means that the observation vector is retrieving information from a previous existing run; in that case, it may not be necessary to dump files

Returns:

a dictionary with structure the components/tracers to be extracted

Return type:

dict

Note

The input data data2dump has a dictionary structure with two levels: component/tracer and date. This reads as:

data2dump = {
    (comp1, tracer1): {
        dd0: pd.DataFrame
        dd1: pd.DataFrame
        [...]
    }
}

In the output, the date level should be removed and only the outputs corresponding to the present simulation (di) should be included

pycif.plugins.models.template.io.outputs2native_adj.outputs2native_adj(self, data2dump, input_type, di, df, runsubdir, mode='fwd', onlyinit=False, do_simu=True, check_transforms=False, **kwargs)[source]#

Dumps and/or save information about outputs, so the model knows where to extract information.

In the present template, observations are simply saved for later use by outputs2native. If the model needs information to extract concentration on-the-fly, the information in data2dump should be used. In particular, the columns i and j are the row and columns of each observation in the domain. The column tstep indicates on which time stamp the observation spans, relative to what is indicated in the variable output_intervals in ini_mapper.

The function is called by loadfromoutputs.adjoint.

Parameters:

self – the model itself
data2dump (dict) – a dictionary with concentration data for each component/tracer
input_type (str) – the type of model outputs to be processed; this information is redundant with the components of the data2dump dictionary
di (datetime.datetime) – starting date of the present sub-simulation
df (datetime.datetime) – ending date of the present sub-simulation
runsubdir (str) – path to the present sub-simulation work directory
mode (str) – running mode; one of “fwd”, “tl” and “adj”
onlyinit (bool) – if True, means that the function is called during the initialization process of the observation vector
do_simu (bool) – if False, means that the observation vector is retrieving information from a previous existing run; in that case, it may not be necessary to dump files

pycif.plugins.models.template.io.inputs.make_auxiliary.make_auxiliary(self, ddi, runsubdir, do_simu=True, mode='fwd', **kwargs)[source]#

Initialize every file or information needed by the model to run, excluding data that are initialized through the function native2inputs.

This includes name lists for Fortran, configuration files, etc.

Every basic files related to the model should be first initialized in self.workdir/model at the initialization step in the function compile.

Hereafter, files are link/copied to runsubdir from the reference ones in self.workdir/model

Note

For configuration files, one should follow the following basic rules:

paths expected by the model should always point to the current runsubdir; thus the executable should be linked or copied in runsubdir; in addition, every extra file should be link with a fixed name and the corresponding name should be given in the name-list or configuration file.

as many model parameters should be easily modified through the yaml configuration file; however, for some reasons, it may be preferable to limit the possibilities for pyCIF by keeping some parameters fixed; this question is up to the developer implementing one model

Parameters:

self – the model plugin
ddi (datetime.datetime) – the start data identifying the present simulation period
runsubdir (str) – path to the current sub-simulation work directory
do_simu (bool) – if False, the simulation does not need to be run, hence, in principle, no auxiliary data needs to be initialized
mode (str) – the running mode to compute

pycif.plugins.models.template — API reference

Contents

pycif.plugins.models.template — API reference#

`pycif.plugins.models.template` — API reference#