Requirements and dependencies in pyCIF

pyCIF automatically links plugins with each other depending on requirements specified in the source of the plugins and on elements of the configuration file.

For instance, the observation operator needs to run the numerical model at some point. For this reason, the model plugin that will be run is attached to the obsoper plugin. In the code of obsoper, the model can thus simply be called as follows:

def obsoper(self):

    # Running the model
    self.model.run(**args, **kwargs)

Another example is a model plugin that needs information about its corresponding domain plugin:

def some-model-method(self):

    # Fetching domain information
    nlon = self.domain.nlon
    nlat = self.domain.nlat

Types of dependencies

There are two main tpes of dependencies in pyCIF:

  1. plugin A calls methods from plugin B

    For that reason, methods must have a standardized format for arguments and outputs, as specified in corresponding pages of the documentation.

  2. plugin A needs data from plugin B

    Similarly, data format needs to follow a specified standardized format

Below is an example graph of dependencies corresponding to the pyCIF configuration file here:

digraph G {
        size ="4,4";
        mode [shape=box, label="Mode:\nforward",style=filled,color=".7 .3 1.0"];   /*this is a comment*/
        model [shape=box, label="Model:\ndummy",style=filled,color=".7 .3 1.0"];   /*this is a comment*/
        controlvect [shape=box, label="Control vector:\nstandard",style=filled,color=".7 .3 1.0"];
        obsoper [shape=box, label="Obs operator:\nstandard",style=filled,color=red];
        obsvect [shape=box, label="Obs vector:\nstandard",style=filled,color=red];
        domain [shape=box, label="Domain:\ndummy",style=filled,color=".7 .3 1.0"];
        meteo [shape=box, label="Meteo:\ndummy",style=filled,color=".7 .3 1.0"];
        measurements [shape=box, label="Measurements:\nrandom",style=filled,color=".7 .3 1.0"];
        fluxes [shape=box, label="Fluxes:\ndummy",style=filled,color=red];

        /*METHOD DEPENDENCIES*/
        edge [style=bold];
        mode -> obsoper;
        obsoper -> obsvect;
        obsoper -> controlvect;
        obsoper -> model;

        /*DATA DEPENDENCIES*/
        edge [color=red];
        obsvect -> measurements;
        controlvect -> model;
        controlvect -> domain;
        obsvect -> model;
        mode -> controlvect;
        obsvect -> measurements;
        model -> meteo;
        model -> domain;
        model -> fluxes;
    }

In this example, blue boxes are plugins that are explicitly defined in the configuration file. Red boxes are implicitly deduced from default values as specified in individual plugins (see here). Black arrows stand for method dependencies, while red ones are data dependencies.

Thus, for instance, the obsoper plugin is required by the mode plugin while it is not specified in the Yaml file. pyCIF automatically initializes the default dependency: obsoper (standard, std).

Behaviour with dependencies

A given plugin might need to use another plugin in different ways, which will determine how the Yaml configuration should be written and how missing requirements will be dealt with. Here are the possible behaviours accepted in pyCIF:

  • any plugin of correct type fits:

XXXXXXXXXX under construction XXXXXXXXXXXXX

Cheat sheet for checking plugins dependencies in yaml files

The following list can be obtained by running the following python commands.

from pycif.utils.classes.baseclass import Plugin
Plugin.print_registered(print_rst=True, print_requirement=True)

List of all available plugins and requirements for each class:

chemistry
controlvect
  • standard, std
    Requires:
    • domain:
      • name: None

      • version: None

      • any: True

      • empty: False

    • model:
      • name: None

      • version: None

      • any: True

      • empty: False

    • datavect:
      • name: standard

      • version: std

      • any: True

      • empty: True

datastream
datavect
  • standard, std
    Requires:
    • domain:
      • name: None

      • version: None

      • any: True

      • empty: False

    • model:
      • name: None

      • version: None

      • any: True

      • empty: False

    • components:
      • name: None

      • version: None

      • any: True

      • empty: True

domain
measurements
minimizer
mode
  • 4dvar, std
    Requires:
    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: False

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • minimizer:
      • name: M1QN3

      • version: std

      • any: True

      • empty: True

    • simulator:
      • name: gausscost

      • version: std

      • any: True

      • empty: True

    • platform:
      • name: None

      • version: None

      • any: True

      • empty: True

  • EnSRF, std
    Requires:
    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: False

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • platform:
      • name: None

      • version: None

      • any: True

      • empty: True

  • adj-tl_test, std
    Requires:
    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: False

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: True

      • empty: True

  • analytic, std
    Requires:
    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: False

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • platform:
      • name: None

      • version: None

      • any: True

      • empty: True

  • footprint, std

  • forward, std
    Requires:
    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: False

      • empty: True

  • post-proc, std
    Requires:
    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • obsoperator:
      • name: standard

      • version: std

      • any: True

      • empty: True

model
  • CHIMERE, std
    Requires:
    • domain:
      • name: CHIMERE

      • version: std

      • any: False

      • empty: False

    • chemistry:
      • name: CHIMERE

      • version: gasJtab

      • any: False

      • empty: False

    • flux:
      • name: CHIMERE

      • version: AEMISSIONS

      • any: False

      • empty: True

    • bioflux:
      • name: CHIMERE

      • version: AEMISSIONS

      • any: False

      • empty: True

    • meteo:
      • name: CHIMERE

      • version: std

      • any: False

      • empty: False

    • latcond:
      • name: CHIMERE

      • version: icbc

      • any: False

      • empty: True

    • topcond:
      • name: CHIMERE

      • version: icbc

      • any: False

      • empty: True

    • inicond:
      • name: CHIMERE

      • version: icbc

      • any: False

      • empty: True

  • FLEXPART, std
    Requires:
    • domain:
      • name: FLEXPART

      • version: std

      • any: False

      • empty: False

    • flux:
      • name: FLEXPART

      • version: nc

      • any: False

      • empty: True

  • LMDZ, std
    Requires:
    • domain:
      • name: LMDZ

      • version: std

      • any: False

      • empty: False

    • flux:
      • name: LMDZ

      • version: sflx

      • any: False

      • empty: True

    • chemistry:
      • name: CHIMERE

      • version: gasJtab

      • any: False

      • empty: False

    • emis_species:
      • name: LMDZ

      • version: bin

      • any: False

      • empty: True

    • meteo:
      • name: LMDZ

      • version: mass-fluxes

      • any: False

      • empty: True

    • inicond:
      • name: LMDZ

      • version: ic

      • any: False

      • empty: True

    • prescrconcs:
      • name: LMDZ

      • version: prescrconcs

      • any: False

      • empty: True

    • kinetic:
      • name: LMDZ

      • version: photochem

      • any: False

      • empty: True

    • prodloss3d:
      • name: LMDZ

      • version: prodloss3d

      • any: False

      • empty: True

  • TM5, std
    Requires:
    • domain:
      • name: dummy

      • version: std

      • any: False

      • empty: False

    • chemistry:
      • name: TM5

      • version: SINK-TIPP

      • any: False

      • empty: False

    • flux:
      • name: TM5

      • version: std

      • any: False

      • empty: True

    • meteo:
      • name: TM5

      • version: std

      • any: False

      • empty: True

    • inicond:
      • name: TM5

      • version: ic

      • any: False

      • empty: True

  • dummy, std
    Requires:
    • domain:
      • name: dummy

      • version: std

      • any: False

      • empty: False

    • flux:
      • name: dummy

      • version: nc

      • any: False

      • empty: True

    • meteo:
      • name: dummy

      • version: csv

      • any: False

      • empty: True

obsoperator
  • FLEXINVERT, std
    Requires:
    • model:
      • name: None

      • version: None

      • any: True

      • empty: False

    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

  • standard, std
    Requires:
    • model:
      • name: None

      • version: None

      • any: True

      • empty: False

    • obsvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • controlvect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • datavect:
      • name: standard

      • version: std

      • any: True

      • empty: True

    • platform:
      • name: None

      • version: None

      • any: True

      • empty: True

obsparser
obsvect
  • standard, std
    Requires:
    • model:
      • name: None

      • version: None

      • any: True

      • empty: False

    • datavect:
      • name: standard

      • version: std

      • any: True

      • empty: True

platform
simulator
transform