.. _chimere-design-yaml-ready-made: Elaborate the YAML for the CIF, using ready-made files ====================================================== .. important:: **How to use the** :doc:`cheat-sheet` **for plugins** The following sections require using plugins and providing their specifications. Arguments for each plugin are documented on its individual documentation page. To make finding plugins easier, the cheat-sheet organizes them by **type** (the leftmost column, e.g. chemistry, controlvect, fields). For each type, available plugins are listed with their **name** and **version**. Specifying a plugin's name and version is mandatory; specifying its type is not always necessary. .. contents:: :local: Section for PyCIF parameters: ----------------------------- This section must contain the five arguments shown in the example: - :bash:`verbose` controls the verbosity level: 1 for basic information, 2 for debugging - :bash:`workdir` is the working directory. The CIF will create it and use it to run the simulation and store all inputs and outputs. Choose a location with sufficient disk space. - :bash:`logfile` is the name of the log file written by the CIF, saved in :bash:`workdir`. - :bash:`datei` and :bash:`datef` are the start and end dates of the simulation period. Accepted formats: ``YYYY-mm-dd`` or ``YYYY-mm-dd HH:mm:ss``. .. container:: toggle .. yml-block:: /yaml-examples/chimere/config_fwd_ref_chimere.yml :keys: verbose, logfile, workdir, datei, datef This section of the YAML can also define :doc:`anchors` for use elsewhere in the file. .. include:: ../../first-simu/common/mode.rst .. _obsoperator_chimere_tuto: .. include:: ../../first-simu/common/obsoperator.rst .. _controlvect_chimere_tuto: .. include:: ../../first-simu/common/controlvect.rst .. _yml_section_model_chimere_fwd: Model (:bash:`model`) --------------------- Use the :doc:`CHIMERE model plugin`. The typical configuration choices for a CHIMERE simulation (see :doc:`CHIMERE documentation`) are split between mandatory and optional arguments; default values are provided for optional ones. Check both the mandatory **and** optional arguments to configure the simulation as intended. The model settings must be consistent with the chemistry (see sections :ref:`step_2_chimere_readymade` and :ref:`chimere_readymade_chemistry`) and the domain (see sections :ref:`step_2_chimere_readymade` and :ref:`chimere_readymade_domain`). .. container:: toggle .. yml-block:: /yaml-examples/chimere/config_fwd_ref_chimere.yml :keys: model The requirements for CHIMERE are :bash:`domain`, :bash:`chemistry` and a set of components (corresponding to the inputs of CHIMERE itself) to be detailed in :bash:`datavect`: :bash:`meteo`, :bash:`flux`, :bash:`bioflux`, :bash:`latcond`, :bash:`topcond` and :bash:`inicond`. Observation Vector (:bash:`obsvect`) ------------------------------------- To avoid unnecessary computations, the CIF only runs the simulation up to the time covered by available observations. The standard :doc:`obsvect` must therefore be initialized. See section :ref:`chimere_readymade_dummy_obs` for how to provide minimal dummy observation data. .. container:: toggle .. yml-block:: /yaml-examples/chimere/config_fwd_ref_chimere.yml :keys: obsvect Its requirements are :bash:`datavect` and the :bash:`model`. .. include:: ../../first-simu/common/platform.rst .. _chimere_readymade_domain: Domain (:bash:`domain`) ------------------------------------- Specify :doc:`a domain for CHIMERE` (see also the :doc:`cheat-sheet`) **consistently** with the pre-computed input files (see step 2). The files defining the domain can be stored directly in directory :bash:`repgrid` or symbolic links can be used. .. container:: toggle .. code-block:: yaml domain : plugin: name : CHIMERE version : std repgrid: a_path_for_CHIMERE_COORD_definition_files/ domid : MYDOMAIN nlev: 20 p1: 997 pmax: 200 pressure_unit: hPa .. _chimere_readymade_chemistry: Chemistry (:bash:`chemistry`) ------------------------------------- The only available chemical scheme type is :doc:`photolysis with tabulated Js`, with the scheme pre-computed (see step 2). .. container:: toggle .. code-block:: yaml chemistry : plugin: name: CHIMERE version: gasJtab schemeid: name.chemistry dir_precomp: the_path_for_the_directory_of_which_chemical_scheme_named_above_is_a_subdir/ Data vector (:bash:`datavect`) ------------------------------------- The :doc:`data vector` contains the input data for the model (e.g. emission fluxes) and for comparison to observations (e.g. concentration data), used by :bash:`controlvect`, :bash:`obsoperator`, and :bash:`obsvect` to build the run configuration. Currently, only the standard :bash:`datavect` plugin is available. .. container:: toggle .. code-block:: yaml datavect : plugin: name: standard version: std For the first forward simulation, its components are the :doc:`requirements of the model plugin` not already covered in earlier YAML sections (i.e. excluding :bash:`domain` and :bash:`chemistry`), plus a component for the (dummy) observation data. .. _chimere-datavect-infos: CHIMERE usual inputs .................... The components that provide the model with its inputs must be chosen from the available :doc:`datastreams` recognized by the model plugin, which uses them to pre-process or fetch its inputs. For CHIMERE, the plugin expects :bash:`flux` to provide hourly-averaged emission data (i.e. data for the AEMISSIONS file). For each datastream component, :bash:`datavect` requires at least three pieces of information: i) :bash:`dir`: the directory where the component's data files are located (mandatory) ii) :bash:`file`: either a fixed filename or a pattern for a set of files using date placeholders (year, month, day, hour, etc.) (mandatory) iii) :bash:`file_freq`: the time span covered by each file, in pandas duration format, e.g. ``1D``, ``120H`` (mandatory, except for initial conditions) To distinguish boundary condition types (lateral, top, initial), :bash:`comp_type` must also be specified. These arguments belong to :bash:`datavect`, not to any particular plugin (see also optional general arguments :doc:`here `). When a plugin is used by a component to process the specified files, its **name, version, and type** must be specified, along with its own arguments. The datastream components for model inputs point to plugins that handle :doc:`the inputs required by CHIMERE`: :doc:`meteo` (I) for meteorological inputs, :doc:`emission fluxes` (II and III), :doc:`boundary conditions` (IV), and :doc:`initial conditions` (IV). I. With pre-computed :doc:`METEO.nc files`, specifying the :bash:`meteo` component is straightforward: i) provide the minimum information: :bash:`dir` and :bash:`file` pointing to ready-made METEO.nc files, and the matching :bash:`file_freq`. ii) add the :doc:`plugin for CHIMERE's ready-made meteo files` (see also the :doc:`cheat-sheet`) .. container:: toggle .. yml-block:: /yaml-examples/chimere/config_fwd_ref_chimere.yml :keys: datavect/components/meteo In this case, METEO.nc filenames follow the :doc:`format used by CHIMERE`. A custom template is also possible, e.g. :bash:`file: METEO_some_etiket.%Y%m%d%H.X.nc`. II. Pre-computed files for fluxes, biofluxes, inicond, latcond, and topcond are specified in the same way. .. note:: All pre-computed files must be mutually consistent and consistent with the domain, the pyCIF parameters, and the model settings, in particular the optional argument :bash:`periods` (default: ``1D`` = 24 hours). Following the same pattern as for meteorology, pre-computed :doc:`AEMISSIONS.nc files` are specified with :bash:`dir` and :bash:`file`, handled by the :doc:`plugin for CHIMERE fluxes`. Use the :doc:`cheat-sheet` to find the correct type, name, and version: .. container:: toggle .. code-block:: yaml datavect: plugin: name: standard version: std components: meteo: dir: directory_containing_METEO.YYYYMMDDHH.*.nc_files file: METEO.%Y%m%d%H.X.nc plugin: name: CHIMERE version: std type: meteo file_freq: XH flux: dir: directory_containing_AEMISSIONS.YYYYMMDDHH.*.nc_files file: AEMISSIONS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: AEMISSIONS type: flux file_freq: XH The CIF expects :doc:`AEMISSIONS.nc files in the exact format used by CHIMERE`, containing all species listed in :doc:`ANTHROPIC`. It is also possible to combine different ready-made files for species that do not require sub-hourly interpolation. These species must be listed as :bash:`parameters` of :bash:`flux`, with names matching those in :doc:`ANTHROPIC`. Each parameter can be configured independently, as shown in :doc:`the tutorial for more elaborated inputs`. III. The :bash:`bioflux` component follows the same principles as :bash:`flux`. If no sub-hourly flux interpolation is needed (:bash:`useemisb` is ``False`` in :bash:`model`), this component can be omitted. If it is omitted while :bash:`useemisb` is ``True``, an error is raised. Pre-computed :doc:`BEMISSIONS.nc files` are specified in the same way as AEMISSIONS, using the :doc:`plugin for CHIMERE fluxes`: .. container:: toggle .. code-block:: yaml datavect: plugin: name: standard version: std components: meteo: dir: directory_containing_METEO.YYYYMMDDHH.*.nc_files file: METEO.%Y%m%d%H.X.nc plugin: name: CHIMERE version: std type: meteo file_freq: XH flux: dir: directory_containing_AEMISSIONS.YYYYMMDDHH.*.nc_files file: AEMISSIONS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: AEMISSIONS type: flux file_freq: XH bioflux: dir: directory_containing_BEMISSIONS.YYYYMMDDHH.*.nc_files file: BEMISSIONS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: AEMISSIONS type: flux emis_type: bio file_freq: XH Note that :bash:`emis_type` must be specified explicitly so that BEMISSIONS files are fetched instead of AEMISSIONS files. When combining files for different parameters, their names must match :doc:`BIOGENIC`. IV. The :bash:`inicond`, :bash:`latcond`, and :bash:`topcond` components are distinguished by their :bash:`comp_type`; otherwise their specifications follow the same principles as :bash:`flux`. Pre-computed :doc:`INI_CONCS.nc files` and :doc:`BOUN_CONCS.nc files` are specified in the same way as AEMISSIONS, using the :doc:`plugin for CHIMERE fields`: .. container:: toggle .. code-block:: yaml datavect: plugin: name: standard version: std components: meteo: dir: directory_containing_METEO.YYYYMMDDHH.*.nc_files file: METEO.%Y%m%d%H.X.nc plugin: name: CHIMERE version: std type: meteo file_freq: XH flux: dir: directory_containing_AEMISSIONS.YYYYMMDDHH.*.nc_files file: AEMISSIONS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: AEMISSIONS type: flux file_freq: XH inicond: dir: directory_containing_IC.YYYYMMDDHH.*.nc_files file: INI_CONCS.0.nc XX mandatory name???XXXX plugin: name: CHIMERE version: icbc type: field comp_type: inicond latcond: dir: directory_containing_BC.YYYYMMDDHH.*.nc_files file: BOUN_CONCS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: icbc type: field file_freq: XH comp_type: latcond topcond: dir: directory_containing_BC.YYYYMMDDHH.*.nc_files file: BOUN_CONCS.%Y%m%d%H.X.nc plugin: name: CHIMERE version: icbc type: field file_freq: XH comp_type: topcond In this configuration, all species listed in :doc:`ACTIVE_SPECIES` are read from the same input files. To use different files for different species, see :doc:`the tutorial for more elaborated inputs`. .. _chimere_readymade_dummy_obs: Component for observations .......................... There are three options for providing observation data to drive CHIMERE. 1. Force computation of the observation operator without real observations (under construction). 2. Generate random observations: specify random surface measurements for a set of parameters using the :doc:`measurements plugin` in the YAML. For example, for a single measured species: .. container:: toggle .. code-block:: yaml concs: parameters: S1: plugin: name: random type: measurements version: param frequency: '1h' nstations: 5 duration: '1h' random_subperiod_shift: True zmax: 100 seed: True 3. Provide your own observation file. The :bash:`concs` component is used for surface data; other types are available as described in the :doc:`standard data vector`. Below is an example for surface observations, including the matching YAML snippet and a Python script to generate a minimal :bash:`monitor.nc` file. .. container:: toggle .. code-block:: yaml concs: parameters: S1: dir: /tmp/PYCIF_DATA_TEST/CHIMERE/ACADOK file: dummy_monitor.nc Parameter names must match entries in the ACTIVE_SPECIES file. To simply run a forward simulation, the dummy monitor file needs only one observation at the final hour of the simulation period. The following short Python script generates such a file: .. container:: toggle .. code-block:: python import pandas as pd import datetime # Put here the elements of datef yearf = 2011 monthf = 2 dayf = 3 hourf = 0 minutef = 0 secondf = 0 # Put here the coordinates of any point in the domain (see repgrid in domain, file HCOORD) lat0 = 1.2 lon0 = 48.3 list_basic_cols = [ 'date', 'duration', 'station' , 'network', 'parameter', 'lon', 'lat', 'obs', 'obserror', 'alt' ] datef = datetime.datetime( year = yearf, month = monthf , day = dayf , hour = hourf, minute = minutef, second = secondf) data = pd.DataFrame( columns = list_basic_cols ) data['date'] = [ datef ] data = data.assign(duration=1.) data = data.assign(station='dummy') data = data.assign(network='none') data = data.assign(parameter='NONE') data = data.assign(lon = lon0) data = data.assign(lat = lat0) data = data.assign(obs = 500. ) data = data.assign(obserror = 5.) data = data.assign(alt =1.) data = data.to_xarray() data.to_netcdf('monitor_obs_for_simu_ID.nc') .. See also :doc:`First simulation with comparison to observations`. .. warning:: when no observation is available, no error is raised, the CIF indicates that the forward mode has been successfully executed - even though CHIMERE did not actually run.