General principles of using transformations: examples for building the inputs of a CTM and/or elaborating on its outputs

Main principles

The components in the datavect do not have to be directly linked to the inputs or outputs of the CTM driven by the CIF.

Various elementary transformations are availble and can be combined by the user to go from the components to the actual inputs of the CTM or to combine the outputs of the CTM to obtain equivalents of the observation data. The personalized (combinations of) transformations is done through the controlvect for dealing with the inputs of the CTM (more generally, for transformations made at the beginning of the CIF’s assimilation chain) and through obsvect for dealing with the outputs of the CTM (more generally, for transormations made at the end of the CIF’s assimilation chain). The transformations are defined by specifying the relevant transform_pipe.

Main steps for defining personalized transformations

For building the inputs of the CTM

We take here the example of fluxes for a forward simulation with CHIMERE.

  1. In the datavect, define the component that contains the original files to use, with the usual information on how to interpolate them on the CTM grid.

In the following example, two raw sources of information are used, for two sectors emitting methane.

datavect:
  plugin:
    name: standard
    version: std

  components:
    meteo:
      plugin:
        name: CHIMERE
        version: std
        type: meteo
      dir: /home/chimereicos/espigrad_reanalysis/inputs/meteo_from_old/
      file: METEO.%Y%m%d%H.24.nc
      file_freq: 1D

    flux:
      parameter:
        CO2:
          plugin:
            name: CHIMERE
            version: AEMISSIONS
            type: flux
          dir: /home/chimereicos/espigrad_reanalysis/inputs/emis_from_independent/
          file: AEMISSIONS.2019%m28%H.24.nc
          file_freq: 1D

    sectorflux:
      parameters:
        CH4emisS10:
          plugin:
            name: CHIMERE
            version: AEMISSIONS
            type: flux
          dir: /home/chimereicos/espigrad_reanalysis/inputs/emis_from_independent/
          file: LNAEMISSIONS.2019%m28%H.24.nc
          file_freq: 1D
          varname: LOGSNAP10
        CH4emisS9:
          plugin:
            name: EDGAR
            version: v5
            type: flux
          dir: /tmp/PYCIF_DATA_TEST/RAW/EMISSIONS/EDGARV5/TOTAL
          regrid:
            method: mass-conservation
          time_interpolation:
            method: linear
          unit_conversion:
            scale: 1.368e+21
          vertical_interpolation:
            method: closest
            fill_nans: false
            fill_nans_value: 0
          varname: emi_ch4_snap9
          file: 'v50_N2O_%Y.0.1x0.1.nc'

Note that:

  • in the example, CO2 is ready-made, CH4 will have to be built from the component sectorflux.

  • the name of the component(s) can be freely defined by the user

  • the parameters’s names can be freely defined by the user

  1. In the controlvect, add the transform_pipe in which the transformations will be built.

controlvect:
  plugin:
    name: standard
    version: std
    type: controlvect
  save_out_netcdf: True
  transform_pipe:
      TBC

  1. Use the elementary transformations to specify the (chain of) personalized transformations to obtain the actual inputs of the CTM from datavect’s components.

controlvect:
  plugin:
    name: standard
    version: std
    type: controlvect
  save_out_netcdf: True
  transform_pipe:
    take_exp: # take the exponential of emissions provided as ln
      plugin:
        name: exp
        type: transform
        version: std
      component: sectorflux
      parameters_in:
      - CH4emisS10
    take_sum_emis: # sum various sectors to obtain the total
      plugin:
        name: families
        version: std
        type: transform
      component: sectorflux
    parameters_in: [ CH4emisS10, CH4emisS9]
    component_out: flux
    parameter_out: CH4

Note

The CIF performs the transformations in the order given in the yaml file. Therefore, in the example, CH4emisS10 is exponentialled then the resulting values are summed up with CH4emisS9.

Note that:

  • the names of the transformations are freely defined by the user (good practice: choose names which are as explicit as possible)

  • the arguments required by each type of transformation are found here

  • the component indicates the component to use in input of the transformation, it must therefore match a name used in datavect

  • the list of components after applying the transform pipe must match the list of required inputs of the CTM so that the names of component_out at the end of a chain of transformations must match the names of components relevant for the CTM

  1. Run XX with only_init?XX and check with the files for checks that the personalized transforms in controlvect do what is expected.

  2. Run the full forward.XX wchich checks?XX

For combining the outputs of the CTM

We take here the example of isotopic data for a forward simulation.

  1. In the datavect, define the component that contains the files of observed data to compare to the simulation.

In the following example, isotopic data of methane are used. They are already formatted into an input monitor file for the CIF.

datavect:
  plugin:
    name: standard
    version: std
  components:
  concs:
    dir: ${PYCIF_DATATEST}/LMDZ//clumped_isotopes//monitor/
    file: monitor_filtered_rsd_new.nc
    file_statlev: ${PYCIF_DATATEST}/LMDZ//simple/stations_levels_LMDZ39lev.txt
    parameters:
      CH4: {}
      d13C: {}

Note that :XXXX what are the parameters???XXX

  1. In the obsvect, add the transform_pipe in which the transformations will be built.

obsvect:
  plugin:
    name: standard
    version: std
  dump: True
  transform_pipe:
      TBC

  1. Use the elementary transformations to specify the (chain of) personalized transformations to obtain the actual simulated equivalents of the observation data from the CTM’s outputs.

obsvect:
  plugin:
    name: standard
    version: std
  dump: True
  transform_pipe:
      makeiso:
          plugin:
              name: conc2ratio
              type: transform
              version: std
       component:
       - concs
       - concs
            XXXXXXXXXXXXXXXXXXXXXXXXX
         parameters_in:
      names:
      - 12CH4
      - 13CH4
    parameters_out:
      d13C:
        isotopologues:
        - 13CH4
        refs:
        - 12CH4
        standard: 0.0112372
      iso_mass:
      - 16.031
      - 17.035
      names:
      - d13C
      - CH4
      spec_mass: 16.0425

    unit: volume