########################################################################################################################## General principles of using transformations: examples for building the inputs of a CTM and/or elaborating on its outputs ########################################################################################################################## Main principles ---------------- The components in the :bash:`datavect` do not have to be directly linked to the inputs or outputs of the CTM driven by the CIF. Various :doc:`elementary transformations` are availble and can be combined by the user to go from the components to the actual inputs of the CTM or to combine the outputs of the CTM to obtain equivalents of the observation data. The personalized (combinations of) transformations is done through the :bash:`controlvect` for dealing with the inputs of the CTM (more generally, for transformations made at the beginning of the CIF's assimilation chain) and through :bash:`obsvect` for dealing with the outputs of the CTM (more generally, for transormations made at the end of the CIF's assimilation chain). The transformations are defined by specifying the relevant :bash:`transform_pipe`. Main steps for defining personalized transformations ----------------------------------------------------- For building the inputs of the CTM ................................... We take here the example of fluxes for a forward simulation with CHIMERE. 1. In the :bash:`datavect`, define the :bash:`component` that contains the original files to use, with the usual information on how to interpolate them on the CTM grid. In the following example, two raw sources of information are used, for two sectors emitting methane. .. code-block:: yaml datavect: plugin: name: standard version: std components: meteo: plugin: name: CHIMERE version: std type: meteo dir: /home/chimereicos/espigrad_reanalysis/inputs/meteo_from_old/ file: METEO.%Y%m%d%H.24.nc file_freq: 1D flux: parameter: CO2: plugin: name: CHIMERE version: AEMISSIONS type: flux dir: /home/chimereicos/espigrad_reanalysis/inputs/emis_from_independent/ file: AEMISSIONS.2019%m28%H.24.nc file_freq: 1D sectorflux: parameters: CH4emisS10: plugin: name: CHIMERE version: AEMISSIONS type: flux dir: /home/chimereicos/espigrad_reanalysis/inputs/emis_from_independent/ file: LNAEMISSIONS.2019%m28%H.24.nc file_freq: 1D varname: LOGSNAP10 CH4emisS9: plugin: name: EDGAR version: v5 type: flux dir: /tmp/PYCIF_DATA_TEST/RAW/EMISSIONS/EDGARV5/TOTAL regrid: method: mass-conservation time_interpolation: method: linear unit_conversion: scale: 1.368e+21 vertical_interpolation: method: closest fill_nans: false fill_nans_value: 0 varname: emi_ch4_snap9 file: 'v50_N2O_%Y.0.1x0.1.nc' Note that: - in the example, CO2 is ready-made, CH4 will have to be built from the component :bash:`sectorflux`. - the name of the component(s) can be freely defined by the user - the parameters's names can be freely defined by the user 2. In the :bash:`controlvect`, add the :bash:`transform_pipe` in which the transformations will be built. .. code-block:: yaml controlvect: plugin: name: standard version: std type: controlvect save_out_netcdf: True transform_pipe: TBC 3. Use the elementary transformations to specify the (chain of) personalized transformations to obtain the actual inputs of the CTM from :bash:`datavect`'s components. .. code-block:: yaml controlvect: plugin: name: standard version: std type: controlvect save_out_netcdf: True transform_pipe: take_exp: # take the exponential of emissions provided as ln plugin: name: exp type: transform version: std component: sectorflux parameters_in: - CH4emisS10 take_sum_emis: # sum various sectors to obtain the total plugin: name: families version: std type: transform component: sectorflux parameters_in: [ CH4emisS10, CH4emisS9] component_out: flux parameter_out: CH4 .. note:: The CIF performs the transformations in the order given in the yaml file. Therefore, in the example, CH4emisS10 is exponentialled **then** the resulting values are summed up with CH4emisS9. Note that: - the names of the transformations are freely defined by the user (good practice: choose names which are as explicit as possible) - the arguments required by each type of transformation are found :doc:`here` - the :bash:`component` indicates the component to use in input of the transformation, it must therefore match a name used in :bash:`datavect` - the list of components after applying the :bash:`transform pipe` must match the list of required inputs of the CTM so that the names of :bash:`component_out` at the end of a chain of transformations must match the names of components relevant for the CTM 4. Run XX with :bash:`only_init`?XX and check with the :doc:`files for checks` that the personalized transforms in controlvect do what is expected. 5. Run the full forward.XX wchich checks?XX For combining the outputs of the CTM ..................................... We take here the example of isotopic data for a forward simulation. 1. In the :bash:`datavect`, define the :bash:`component` that contains the files of observed data to compare to the simulation. In the following example, isotopic data of methane are used. They are already formatted into an input monitor file for the CIF. .. code-block:: yaml datavect: plugin: name: standard version: std components: concs: dir: ${PYCIF_DATATEST}/LMDZ//clumped_isotopes//monitor/ file: monitor_filtered_rsd_new.nc file_statlev: ${PYCIF_DATATEST}/LMDZ//simple/stations_levels_LMDZ39lev.txt parameters: CH4: {} d13C: {} Note that :XXXX what are the parameters???XXX 2. In the :bash:`obsvect`, add the :bash:`transform_pipe` in which the transformations will be built. .. code-block:: yaml obsvect: plugin: name: standard version: std dump: True transform_pipe: TBC 3. Use the elementary transformations to specify the (chain of) personalized transformations to obtain the actual simulated equivalents of the observation data from the CTM's outputs. .. code-block:: yaml obsvect: plugin: name: standard version: std dump: True transform_pipe: makeiso: plugin: name: conc2ratio type: transform version: std component: - concs - concs XXXXXXXXXXXXXXXXXXXXXXXXX parameters_in: names: - 12CH4 - 13CH4 parameters_out: d13C: isotopologues: - 13CH4 refs: - 12CH4 standard: 0.0112372 iso_mass: - 16.031 - 17.035 names: - d13C - CH4 spec_mass: 16.0425 unit: volume