.. role:: bash(code) :language: bash ###################### Models (:bash:`model`) ###################### .. contents:: Contents :local: Available Models (:bash:`model`) ================================= The following :bash:`models` are implemented in pyCIF so far: .. toctree:: chimere chimere_acc iconart lmdz lmdz_acc lagrangian TM5 template dummy wrfchem Description =========== The :bash:`model` class runs chemistry-transport models, process their outputs and generates their inputs. Please note that models are often computed with high-performance languages such as Fortran or C. In these case, the sources are included in the directory ``model_sources`` provided alongside pyCIF. Required parameters, dependencies and functions =============================================== The following attributes, dependencies and functions should be defined for any :bash:`model`, as they are called by other plugins. They can be parameters to define at the set-up step, functions to implement in the corresponding module, or dependencies to be attached to the :bash:`model` class. Parameters and attributes +++++++++++++++++++++++++ Initialization parameters ------------------------- The following attributes are defined once for all at the initialization of the model, they inform pyCIF about the temporal resolution of the model. All the following objects are filled with `datetime.datetime `__ objects. To make the handling of lists easier, pyCIF requires lists to be implemented as `numpy.array `__ Below, only ``subsimu_dates`` is mandatory, the others are recommended to be called elsewhere, in particular for the definition of the ``mapper``. :``subsimu_dates``: the list of simulation periods if the model simulation window is split into shorter sub-periods :``tstep_dates``: the time-steps at which the model carries out its numerical computations; this argument is used by pyCIF to determine which observation to compare to what model time step. The shape of this argument is a dictionary, whose keys are ``subsimu_dates`` and entries are the lists of time-steps corresponding to each sub-period. :``tstep_all``: the same as ``tstep_dates``; the difference is that ``tstep_all`` is a list containing ``all`` time steps of all simulation sub-periods instead of a dictionary split into sub-periods :``input_dates``: dates at which the model expects some inputs; has the same shape as ``tstep_dates`` Please find below an illustration of the different time steps: .. graphviz:: ../../time_steps.dot :align: center In the example, the model is run between January 1st, 2010 to February 28th, 2010. Computations are carried out every hours and inputs are expected every 3 hours. In that case, the temporal variables are: .. code-block:: python import numpy as np subsimu_dates = np.array([datetime.datetime(2010, 1, 1), datetime.datetime(2010, 2, 1)]) tstep_dates = { datetime.datetime(2010, 1, 1): np.array( [datetime.datetime(2010, 1, 1, 0), datetime.datetime(2010, 1, 1, 1), ..., datetime.datetime(2010, 1, 31, 23)]), datetime.datetime(2010, 2, 1): np.array( [datetime.datetime(2010, 2, 1, 0), datetime.datetime(2010, 2, 1, 1), ..., datetime.datetime(2010, 2, 28, 23)]), } tstep_all = np.array([ datetime.datetime(2010, 1, 1, 0), datetime.datetime(2010, 1, 1, 1), ..., datetime.datetime(2010, 2, 28, 23) ]) input_dates = { datetime.datetime(2010, 1, 1): np.array( [datetime.datetime(2010, 1, 1, 0), datetime.datetime(2010, 1, 1, 3), ..., datetime.datetime(2010, 1, 31, 21)]), datetime.datetime(2010, 2, 1): np.array( [datetime.datetime(2010, 2, 1, 0), datetime.datetime(2010, 2, 1, 3), ..., datetime.datetime(2010, 2, 28, 21)]), } Online parameters ----------------- The following variables are defined online during the computation of the model. :``chain``: for a given model simulation, files from previous sub-periods necessary to run following sub-periods are stored in ``current_sim_directory/chain``; the ``chain`` variable stores the date of the previous sub-period that was computed; the variable is automatically updated by the :bash:`obsoperator`, but the files should be moved by the function :bash:`run` of the model. :``adj_refdir``: this is the directory where forward simulations corresponding to the adjoint being run are stored; the variable should be updated when running a forward in the :bash:`run` function. Dependencies ++++++++++++ Some other classes in pyCIF expect the :bash:`model` class to have a :bash:`domain` class attached to it, describing the model domain. This way, :bash:`model.domain` can be called. Functions +++++++++ The following functions need to be implemented in any model to make it interact with other classes. They must be imported at the root level of the corresponding python package, i.e. in the ``__init__.py`` file: .. code-block:: python from XXXXX import ini_periods from XXXXX import run from XXXXX import make_auxiliary from XXXXX import native2inputs from XXXXX import native2inputs_adj from XXXXX import outputs2native from XXXXX import outputs2native_adj from XXXXX import compile from XXXXX import ini_mapper It is recommended to include each function in a separate file to avoid very long scripts. ini_periods (optional) ---------------------- .. module:: pycif.plugins.models.template :noindex: .. autofunction:: ini_periods :noindex: Click below to see an example of the :bash:`ini_periods` function for the model CHIMERE. .. module:: pycif.plugins.models.chimere :noindex: .. autofunction:: ini_periods :noindex: run --- The function :bash:`run` executes the model itself. As models are often computationally expensive to run, they are not written in python. Therefore, the :bash:`run` function calls an external executable compiled previously. There are several ways to call system executables in python. We recommend using the function `subprocess.Popen `__ for that purpose. It gives flexibility in logging and can capture errors during the execution of the external executable. Other tasks carried out by the :bash:`run` function are: * update the variable :bash:`self.adj_refdir` for later adjoint simulations * update the variable :bash:`self.chain` for later sub-periods and move necessary files to that directory; these files include for instance concentration fields at the last time step of the period, to be used as initial conditions for the next period. .. module:: pycif.plugins.models.template :noindex: .. autofunction:: run :noindex: Click below for a full example of the :bash:`run` function for the model CHIMERE. .. module:: pycif.plugins.models.chimere :noindex: .. autofunction:: run :noindex: ini_mapper ---------- The function ``ini_mapper`` defines the ``mapper`` giving meta-data about the model's inputs and outputs. The ``ini_mapper`` function dedicated to a model has the same structure of the ``ini_mapper`` functions for the ``transform`` Plugins. Please consult the :doc:`corresponding page <../transforms/index>` for further details. outputs2native and outputs2native_adj ------------------------------------- The functions :bash:`outputs2native` and :bash:`outputs2native_adj` read outputs and generate sensitivity to the outputs respectively. .. module:: pycif.plugins.models.template :noindex: .. autofunction:: outputs2native_adj :noindex: .. autofunction:: outputs2native :noindex: native2inputs and native2inputs_adj ----------------------------------- The functions :bash:`native2inputs` and :bash:`native2inputs_adj` generate inputs for the model executable and reads sensitivity to the inputs as computed by the adjoint respectively. .. module:: pycif.plugins.models.template :noindex: .. autofunction:: native2inputs :noindex: .. autofunction:: native2inputs_adj :noindex: make_auxiliary (optional) ------------------------- This function is called at the same time as ``native2inputs``. It generates all required information or files that are not data coming from the ``datavect`` and included in the ``mapper``, hence initialized by ``native2inputs``. Example of code: .. module:: pycif.plugins.models.template :noindex: .. autofunction:: pycif.plugins.models.template.make_auxiliary :noindex: compile (optional) ------------------ The function copies, links or compiles the necessary model executables and, if needed, auxiliary configuration files .. module:: pycif.plugins.models.template :noindex: .. autofunction:: pycif.plugins.models.template.compile :noindex: flushrun (optional) ------------------- The function :bash:`flushrun` is called at the end of a simulations. It cleans all temporary files that take disk space and are not necessary afterwards. Arguments are: :self: the model itself :rundir: the run directory (with all the sub-period simulations) :mode: the running mode; one of ``fwd``, ``tl`` or ``adj``. The function returns nothing. Click below for a full example of the :bash:`flushrun` function for the model CHIMERE. .. module:: pycif.plugins.models.chimere :noindex: .. autofunction:: flushrun :noindex: