Plugins: what are they?¶
Plugins in pyCIF are the elementary block of the python library. They are the basic class on which all other classes are built.
All data and meta-data in pyCIF are stored as attributes of
Plugin
class instance.
Dependencies and links between each plugin
are automatically dealt with,
as further explained here.
The list of classes in the CIF is detailed below. Click to expand the code of each class. For each class, please refer to the corresponding page for the list of available plugins and further documentation:
Plugin
classClick here to expand the code of the
Plugin
class
- class pycif.utils.classes.baseclass.Plugin(plg_orig=None, **kwargs)[source]¶
The Plugin class is the parent class of all other pyCIF classes. It is used to store all information about sub-parts of pyCIF and loads if needed sub-modules.
- classmethod childclass_factory(plg_orig, child_type=None, parent_plg=None, overwrite=False)[source]¶
Generates an instance of one of Plugin’s child classes. Transfers all existing attributes in the argument plugin to the output child-class instance
- Parameters
plg_orig (Plugin) – the plugin to turn into a child-class instance
child_type (str) – sub-class type to generate if not available in plg_orig.plugin.type
overwrite (bool) – overwrite the class type of the origin plugin
- Returns
a plugin with all the attributes from plg_orig, but as a child-class instance
- Return type
child_plg
- classmethod dump_incorrect(file_dump)[source]¶
Dump incorrect arguments from the Yaml into a file
- Parameters
file_dump – File where to dump
- Returns
- classmethod from_dict(def_dict, orig_name='', convert_none=False, **kwargs)[source]¶
Loads a recursive dictionary structure into a Plugin
- Parameters
def_dict (dict) – the definition dictionary
orig_dict (dict) – the definition dictionary used at level 0
- Returns
Plugin
- classmethod from_yaml(def_file)[source]¶
Generates a dictionary including all pyCIF parameters
- Parameters
def_file (string) – Path to the definition file Handles both absolute and relative paths
- Returns
Dictionary populated with all pyCIF parameters
- Return type
config_dict (dictionary)
- classmethod get_loaded(name, version, plugin_type, subtype='')[source]¶
Get the correct loaded plugin, given its name, version and type (and optionally subtype)
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
- Returns
plugin module for plugin version
- Return type
- classmethod get_registered(name, version, plugin_type, subtype='')[source]¶
Get the correct registered plugin, given its name, version and type
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
- Returns
plugin module for plugin version
- Return type
- classmethod get_subclass(plg_type, plg_subtype='')[source]¶
Get the plugin class template from a given type
- Parameters
plg_type (str) – the plugin type to load
plg_subtype (str) – the plugin sub-type to load
- Returns
Empty instance of the correct class type
- initiate(plg_type=None)[source]¶
Initializes a Plugin, i.e., loads functions from registered plugins
- Parameters
plg_type (str) – the type of plugin to load; this should
to one of the defined child-classes (correspond) –
- Returns
a python module as registered in pyCIF
- Return type
module
- initiate_template(plg_type=None, default_functions={})[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod is_allowed(plugin_type)[source]¶
Check whether a plugin type is allowed in pyCIF or not
- Parameters
plugin_type (str) – type of plugin
- Returns
True if allowed plugin
- Return type
bool
- classmethod is_allowed_as_subtype(plugin_type)[source]¶
Check whether a plugin type is allowed as the sub-type of a main type in pyCIF or not
- Parameters
plugin_type (str) – type of plugin
- Returns
True if allowed plugin
- Return type
bool
- classmethod is_loaded(name, version, plugin_type, subtype='')[source]¶
Check whether a plugin is loaded
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
- Returns
bool
- classmethod is_registered(name, version, plugin_type, subtype)[source]¶
Check if a plugin is registered
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
- Returns
True if a parser is registered key: the key of the registered Plugin if registered
- Return type
bool
- classmethod load_registered(name, version, plg_type, plg_subtype='', plg_orig=None)[source]¶
Get a sub-class instance of a registered plugin. This can be used to get default required plugins.
- static plugin_key(name, version, plugin_type, subtype)[source]¶
Creates the key for a name, version and plugin type
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
- Returns
dict key for a plugin and version
- Return type
set
- classmethod print_registered(print_requirement=False, print_rst=False, types=[], names=[], versions=[], stream=None)[source]¶
Print in a user-friendly format the list of available plugins
- Parameters
print_requirement (bool) – for each registered plugin, print its requirements
print_rst (bool) – print in rst format for automatic use in the documentations
types (list) – list of types of Plugins to print
names (list) – list of names of Plugins to print
version (list) – list of versions of Plugins to print
stream – stream to which send the display. By default, stdout is used. A stream to a given file can be used instead.
- classmethod register_plugin(name, version, module, plugin_type='', subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
plugin_type (str) – type of plugin
subtype (str) – sub-type of plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
**kwargs (dictionary) – default options for module
- classmethod save_loaded(plg)[source]¶
Saves all loaded plugins to the class
- Parameters
plg (Plugin) – plugin to save
- set_requirements()[source]¶
Update requirements depending on the Plugin properties
By default, this method does nothing. It can be included in any new Plugin depending on the developer needs.
- Parameters
self (Plugin) – the Plugin to update
Setup
classClick here to expand the code of the
Setup
class
- class pycif.utils.classes.setup.Setup(plg_orig=None, **kwargs)[source]¶
- classmethod load_setup(plg, parent_plg_type=None, level=999, tree='', list_levels=None, ignore_parent_type=False, **kwargs)[source]¶
Loads a Setup plugin. Loops recursively over all attributes of the setup to load: 1) sub-plugins are initialized as Plugin child-class templates ( Domain, ObsVect, Model, etc); 2) instances are saved to the Plugin class to be accessible for anywhere later one.
This allows modifications of the data of a given plugin at some place of the code to be automatically forwarded to the rest of the code
- Parameters
self (Setup) – the setup to load
parent_plg_type (str) – the last recognized plugin type that is
by children (inherited) –
Click here to expand the code of the
Chemistry
class
- class pycif.utils.classes.chemistries.Chemistry(plg_orig=None, **kwargs)[source]¶
- create_chemicalscheme(*args, **kwargs)[source]¶
Creates a chemical scheme if needed
- Parameters
chemistry (dictionary) – dictionary defining the chelical scheme
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- read_chemicalscheme(*args, **kwargs)[source]¶
Read a chemical scheme from an existing file
- Parameters
self (Chemistry) – plugin defining the chemistry. Should include
grid to be able to read the chemistry from files (dirscheme) –
- Returns
Characteristics of the chemical scheme
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Controlvect
class
- class pycif.utils.classes.controlvects.ControlVect(**kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Datastream
class
- class pycif.utils.classes.datastreams.DataStream(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Datavect
class
- class pycif.utils.classes.datavects.DataVect(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Domain
class
- class pycif.utils.classes.domains.Domain(**kwargs)[source]¶
- create_domain(*args, **kwargs)[source]¶
Creates a grid if needed
- Parameters
domain (dictionary) – dictionary defining the domain.
- ini_data(**kwargs)[source]¶
Initializes the domain depending on the model used for the inversion. Defines a domain grid from a grid file or a set of parameters if the domain was not already defined. Domains are model dependant, so the outputs can be different depending on the model.
- Parameters
plugin (DomainPlugin) – domain definition
- Returns
Updates on the fly the domain
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- read_grid(*args, **kwargs)[source]¶
Read a grid from an existing file
- Parameters
self (Domain) – plugin defining the domain. Should include
to be able to read the grid from a file (filegrid) –
- Returns
Grid domain with meshgrids for center lon/lat and corner lon/lat
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Measurement
class
- class pycif.utils.classes.measurements.Measurement(plg_orig=None, **kwargs)[source]¶
- ini_data(**kwargs)[source]¶
Initializes the measurement plugin. This is pased on iterating over multiple species and providers
- Parameters
plugin (MeasurementPlugin) – Measurement definition
- Returns
Updates on the fly the measurements
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Minimizer
class
- class pycif.utils.classes.minimizers.Minimizer(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Model
class
- class pycif.utils.classes.models.Model(**kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Mode
class
- class pycif.utils.classes.modes.Mode(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Obsoperator
class
- class pycif.utils.classes.obsoperators.ObsOperator(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- obsoper(inputs, mode, run_id=0, datei=datetime.datetime(1979, 1, 1, 0, 0), datef=datetime.datetime(2100, 1, 1, 0, 0), workdir='./', **kwargs)[source]¶
The observation operator. This function maps information from the control space to the observation space and conversely depending on the running mode.
- Parameters
self (ObsOperator) – the ObsOperator plugin
inputs (Controlvect or Obsvect) – the inputs of the fwd or adj mode
mode (str) – the running mode
run_id (int) – the ID of the current run (determines the
name (sub-directory) –
datei (datetime.datetime) – beginning of the simulation window
datef (datetime.datetime) – end of the simulation window
workdir (str) – path to the parent directory
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Obsparser
class
- class pycif.utils.classes.obsparsers.ObsParser(plg_orig=None, **kwargs)[source]¶
Class for handling time series parsing from different data providers and data file formats.
- static check_df(df, **kwargs)[source]¶
Check that all required columns have been loaded
- Parameters
df –
kwargs –
- Returns
- classmethod get_parser(plg)[source]¶
Get the correct Parser for a provider and file_format_id
- Parameters
provider (str) – provider of the input file
file_format_id (str) – name of the type of file with a given format
- Returns
Parser for provider and file_format_id
- Return type
Parser
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- parse_file(obs_file, **kwargs)[source]¶
This function does the parsing (and post processing if necessary).
- Parameters
obs_file (str) – path to input file
- Keyword Arguments
encoding (str) – Encoding of input files
freq (str) – frequency after resampling see `Offset Aliases`_ for valid strings
src_freq (str) – explicit setting of the frequency in the input file shouldn’t be necessary
- Returns
renamed, shifted, resampled Dataframe df[obssite_id, parameter] with t as index
- Return type
pandas.DataFrame
- parse_multiple_files(**kwargs)[source]¶
Parses multiple files specified by a glob pattern and stores the content into a datastore.
- Parameters
self – the plugin with its describing arguments (in particular dir_obs)
- Returns
{obs_file} = df[obssite_id, parameter]
- Return type
dict
Note
By default, the function calls self.parse_file, which filters out NaNs and check that all required columns are available.
- classmethod register_parser(provider, file_format_id, parse_module, **kwargs)[source]¶
Register a parsing function for provider and format with default options
- Parameters
provider (str) – provider of the input file
file_format_id (str) – name of the type of file with a given format
parse_module (Module) – returns file content as pandas.DataFrame df[obssite_id, parameter]
**kwargs – default options for parse_function
Notes
The parse_function signature is the same as the
Parser.parse_file()
Click here to expand the code of the
Obsvect
class
- class pycif.utils.classes.obsvects.ObsVect(**kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Platform
class
- class pycif.utils.classes.platforms.Platform(**kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Simulator
class
- class pycif.utils.classes.simulators.Simulator(plg_orig=None, **kwargs)[source]¶
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module
Click here to expand the code of the
Transform
class
- class pycif.utils.classes.transforms.Transform(plg_orig=None, **kwargs)[source]¶
- classmethod get_transform(plg)[source]¶
Get the correct Parser for a provider and file_format_id
- Parameters
provider (str) – provider of the input file
file_format_id (str) – name of the type of file with a given format
- Returns
Parser for provider and file_format_id
- Return type
Parser
- initiate_template()[source]¶
Initializes a Plugin template, with methods from the corresponding module.
- Parameters
self – the plugin to initialize
plg_type – the type of the plugin to initialize
default_functions (dict[str, bool]) – functions to load from the module and to attach to the plugin. Each key names the function and each value is a boolean to determine whether the corresponding function is a class method (with a reflective self as argument) or a classical static function
- mapper2batch(nsamples, transf_mapper, all_mapper, all_transforms, dir_samples, **kwargs)[source]¶
Default mapper2batch method.
Perturb the mapper to simulate a batch of samples at once.
Propagate perturbations from precursors to outputs.
For transformations with no inputs, perturb outputs if “fromcontrol”
- perturb_transform(*args, **kwargs)[source]¶
Default empty perturb_transform method.
Do nothing by default.
If specified in the corresponding plugin, the function should perturb its own behaviour beyond the mapper if necessary.
For instance, for models, it is necessary to update the chemical scheme to know which species should be accounted for
- classmethod register_plugin(name, version, module, subtype='', **kwargs)[source]¶
Register a module for a plugin and version with possibly options
- Parameters
name (str) – name of the plugin
version (str) – version of the plugin
module (types.ModuleType) – module defining the interface between pyCIF and the plugin
plugin_type (str) – type of plugin
**kwargs (dictionary) – default options for module