How to add, register and initialize a new plugin

pyCIF is organized around so-called plugins interacting with each other. Further details about plugins are given here. Here, you will learn how to add a new plugins of any type.

Adding a plugin to pyCIF

A plugin is simply a python module. To add a new plugin of a given class to pyCIF, you need to create a new folder in the correct class folder in pycif/plugins/. You then need to create an empty file called __init__.py, so python interprets the new folder as a python module.

cd pycif/plugins
cd the-class-you-need-to-create
mkdir your-new-plugin
cd your-new-plugin
touch __init__.py

Note

Although the __init__.py is not strictly needed by the newest version of python, pycif fetches information directly from this file, for initializing the plugin and automatically documenting it. Therefore, please include it anyway

Registering your new plugin to pyCIF

pyCIF attaches plugin functions as defined in the corresponding python module in pycif/plugins/ automatically from the yaml configuration file. When the new plugin is created, it must be registered, so it can be called by other routines of pyCIF.

plugins in pyCIF are identified with:

• a name

• optional: a version (default: std)

To register a new plugin, one must define the name and (if relevant) version in the file __init__.py :

_name = "the_name"
_version = "the_version_if_not_std"

You can check that your plugin is correctly registered by using the following commands in python:

from pycif.utils.classes.baseclass import Plugin
Plugin.print_registered()

Adding requirements to your plugin

All plugins are not stand-alone modules, but instead need attributes, data or functions from other plugins. pyCIF allows you to easily interface plugins with each other through so-called requirements. Further details on requirements can be found here.

Information about requirements are specified in the __init__.py file of your plugin. Below is an example of requirements for the model CHIMERE:

requirements = {
  "domain": {
    "name": "CHIMERE",
    "version": "std",
    "empty": False,
    "any": False,
  },
  "chemistry": {
    "name": "CHIMERE",
    "version": "gasJtab",
    "empty": False,
    "any": False,
  },
  "flux": {
    "name": "CHIMERE",
    "version": "AEMISSIONS",
    "empty": True,
    "any": False,
    "subplug": True,
    "preftree": "datavect/components",
  },
[...]
}

With the example above, the model plugin, CHIMERE, requires a domain plugin to work. In the sub-routines of the model plugin, data from the domain can simply be summoned with:

def some_model_function(self, *args, **kwargs):

    # Needs the number of grid cells
    nlon = self.domain.nlon
    nlat = self.domain.nlat

The content of the requirements python dictionary is interpreted when a pyCIF run is initializing. Requirements are fulfilled with regards to the Yaml configuration file. Depending the values of the arguments, name, version, empty any, pyCIF will need the requirement to be explicitly specified or not, default plugins could be used, etc. All details on how this works are given here.

Input arguments for your plugin

Not all plugins have input arguments. Input arguments are variables provided by the user through the yaml defining the simulation.

Input arguments are grouped in a dictionary input_arguments, which contains, for each argument, the documentation, its default value (‘None’ if no default is permitted i.e. the argument must be explicitly defined) and the accepted type(s) or value(s).

input_arguments = {
  "direxec": {
    "doc": "Path to CHIMERE sources and/or executables. "
           "For executables, ``fwdchimere.e``, ``tlchimere.e`` and ``achimere.e`` should be in "
           "``${path}/src``, ``${path}/src_tl`` and ``${path}/src_ad`` respective sub-folders.",
    "default": None,
    "accepted": str
  },
  "ideepconv": {
    "doc": "Computation of the deep convection.",
    "default": None,
    "accepted": {
        0: "No deep convection",
        1: "Select deep convection automatically according to resolution, "
           "deep conv fluxes from Tiedtke",
        2: "Select deep convection automatically according to resolution, "
           "deep conv fluxes from meteorological data"
      }
    },
[...]
}

Initializing your plugin

Your plugin may need some special operations at initialization. For instance, it may need to copy files, to compile scripts, to read data, to specified default values, etc. All these operations can be specified in the __init__.py by creating a function called ini_data.

Below is the ini_data method for the model plugin, CHIMERE:

def ini_data(plugin, **kwargs):
    """Initializes CHIMERE

    Args:
        plugin (dict): dictionary defining the plugin
        **kwargs (dictionary): possible extra parameters

    Returns:
        loaded plugin and directory with executable

    """

    info("Initializing the model")

    workdir = getattr(plugin, 'workdir', './')

    # Initializes the directory
    path.init_dir('{}/model'.format(workdir))

    # Copying the executables
    target = '{}/model/'.format(workdir)

    source = '{}/src/fwdchimere.e'.format(plugin.direxec)
    shutil.copy(source, target)

    source = '{}/src_tl/tlchimere.e'.format(plugin.direxec)
    shutil.copy(source, target)

    source = '{}/src_ad/achimere.e'.format(plugin.direxec)
    shutil.copy(source, target)

    # Required inputs for running a CHIMERE simulation
    plugin.required_inputs = ['exe', 'nml', 'fluxes',
                              'meteo', 'inicond', 'boundcond']

    # Default values:
    # period: '1D'
    plugin.periods = getattr(plugin, 'periods', '1D')

    # Number of hours per period
    plugin.nhours = int(pd.to_timedelta(plugin.periods).total_seconds() / 3600)
    plugin.nho = '{:.0f}'.format(plugin.nhours)

    # Replace name for AEMISSION files
    plugin.fluxes.file = plugin.fluxes.file.format(nho=plugin.nho)

    return plugin

During initialization of the model plugin, CHIMERE, the following operations are carried out:

• creating a new directory model/ in the working directory with CHIMERE executables to be called later for simulations

• specifying a default value for the attribute period if not given in the configuration file

• modifying the template name of CHIMERE fluxes

Attaching functions to the plugin

Any plugin is a combination of functions that will be called by itself or by other plugins. It is necessary to attach functions to your plugin so that they can be called this way in other scripts:

my-plugin.my-function(*args, **kwargs)

To do so, you simply need to import the function of interest at the module level, i.e., in the file __init__.py. For any function of interest the import line should be added to the __init__.py file.

import my-function