Requirements and dependencies in pyCIF
######################################

pyCIF automatically links plugins with each other depending on requirements specified
in the source of the plugins and on elements of the configuration file.

For instance, the observation operator needs to run the numerical model at some point.
For this reason, the :bash:`model` plugin that will be run is attached to the
:bash:`obsoper` plugin.
In the code of :bash:`obsoper`, the model can thus simply be called as follows:

.. code-block:: python

    def obsoper(self):

        # Running the model
        self.model.run(**args, **kwargs)

Another example is a :bash:`model` plugin that needs information about its corresponding :bash:`domain` plugin:

.. code-block:: python

    def some-model-method(self):

        # Fetching domain information
        nlon = self.domain.nlon
        nlat = self.domain.nlat


Types of dependencies
-----------------------

There are two main types of dependencies in pyCIF:

1.  :bash:`plugin A` calls methods from :bash:`plugin B`
        For that reason, methods must have a standardized format for arguments and outputs,
        as specified in the corresponding pages of the documentation.
2.  :bash:`plugin A` needs data from :bash:`plugin B`
        Similarly, data format needs to follow a specified standardized format

Below is an example graph of dependencies corresponding to the pyCIF configuration file :doc:`here</documentation/plugins/dependencies_example>`:

.. image:: dependencies.svg
   :align: center

In this example, blue boxes are :bash:`plugins` that are explicitly defined in the configuration file.
Red boxes are implicitly deduced from default values as specified in individual :bash:`plugins`
(see :doc:`here<available>`).
Black arrows stand for method dependencies, while red ones are data dependencies.

Thus, for instance, the :bash:`obsoper` plugin is required by the :bash:`mode` plugin while it is not specified in the Yaml file.
pyCIF automatically initializes the default dependency: :bash:`obsoper` (standard, std).

Definition of dependencies
---------------------------

Information about :bash:`requirements` are specified in the :bash:`__init__.py` file of your :bash:`plugin`.
To define requirements, one has to include a dictionary called :bash:`requirements` in the :bash:`__init__.py` file.

Keys of the dictionary are other :bash:`plugins` needed for the execution of the parent :bash:`plugin`.
In the example below, the parent :bash:`plugin` will be attached a :bash:`domain` :bash:`plugin` as attribute,
later callable using :bash:`self.domain`.

.. code-block:: python

    requirements = {
        "domain": {
            [...]
        }
    }

.. note::

    The possible keys in each key of the :bash:`requirements` dictionary are detailed below.

    Please note that it is possible to give extra keys that will be used to define
    arguments of the corresponding plugin.

    For instance, in the case above, one can write:

    .. code-block:: python

        requirements = {
            "domain": {
                "some_extra_key": "grub"
            }
        }

    Then, one will be able to call :bash:`self.domain.some_extra_key` in the internal
    functions of the corresponding plugin.

    Moreover, the keys defined in that manner can be used to alter the way the :bash:`domain`
    will initialize itself.

.. warning::

    The following keys are **reserved** by the dependency mechanism and are consumed
    internally.  They will **not** be forwarded as attributes to the attached plugin,
    so they cannot be used as custom extra keys:

    ``name``, ``version``, ``type``, ``any``, ``subplug``, ``preftree``, ``empty``, ``newplg``


Behaviour with dependencies
---------------------------

A given plugin might need to use another plugin in different ways, which will determine how the Yaml configuration
should be written and how missing requirements will be dealt with.
The expected usage of dependencies is determined through the definition of each corresponding key of the dictionary
:bash:`requirements`.

The possible arguments to provide to each key of the dictionary are the following:

    - :bash:`name`/:bash:`version`/:bash:`type`/:bash:`subtype`:
      name/version/type/subtype of the required plugin; type and subtype are optional if the key corresponds to
      a type of plugins.
      For instance, if the key is :bash:`domain`, there is no need to repeat the type.
    - :bash:`any`: any plugin fitting the required type is fine
    - :bash:`subplug`: authorizes to search for required plugins not only at the root level of the yaml.
    - :bash:`preftree` (to be defined if :bash:`subplug` is True): when several plugins fit the requirement,
      select the one whose YAML tree path contains the :bash:`preftree` substring.
      Among matching candidates, the one at the shallowest level (shortest path) is chosen.
      If no candidate matches :bash:`preftree`, pyCIF falls through to the :bash:`empty` fallback rather
      than raising an error.
    - :bash:`empty`: an empty plugin is defined according to the :bash:`name`/:bash:`version`/:bash:`type`/:bash:`subtype`
      if none is available in the Yaml. This can be used when only functions of the required plugin are necessary and no data.
    - :bash:`newplg`: force initializing a new plugin instance instead of reusing the existing one from the Yaml.

The resolution of each requirement is implemented in
:meth:`~pycif.utils.classes.baseclass.Plugin._fetch_requirement`
(see the :doc:`API reference </documentation/classes/classes.baseclass>`).
In practice, it follows a strict **priority chain** (first match wins):

    1. **Child attribute** — a plugin explicitly defined as a child of the parent plugin in the Yaml
       (i.e., a sub-paragraph of the parent's Yaml block).

    2. **Level-0 reference** — a plugin defined at the root level of the Yaml file
       (i.e., a top-level paragraph such as :bash:`model`, :bash:`domain`, etc.).

    3. **Unambiguous sub-reference** (:bash:`subplug: True` only) — if there is exactly one plugin
       of the required type anywhere in the Yaml tree, use it.

    4. **Disambiguated sub-reference** (:bash:`subplug: True` only) — if several candidates exist,
       select the one whose Yaml path best matches :bash:`preftree` (shallowest path wins).

    5. **Empty fallback** (:bash:`empty: True`) — load the registered default plugin of the required
       :bash:`name`/:bash:`version`/:bash:`type`, or instantiate a bare subclass if none is registered.

    6. **Error** — if none of the above conditions can be satisfied, a :class:`PluginError` is raised.

The diagram below summarises this priority chain:

.. image:: fetch_requirement.svg
   :align: center
   :alt: Flowchart of Plugin._fetch_requirement() priority chain

.. note::

    Steps 1 and 2 establish a precedence: a child plugin defined inline under the parent plugin
    in the Yaml **shadows** a top-level plugin of the same type.
    This matters when, for example, a model defines its own :bash:`domain` that differs from a
    domain defined at the root level.

Once a candidate is found, pyCIF decides whether to attach it directly or replace it with
the registered default:

    - If :bash:`any` is True **and** the candidate has a valid plugin identity, attach it as-is.
    - If :bash:`any` is False, attach it only if its :bash:`name`/:bash:`version`/:bash:`type`/:bash:`subtype`
      match the requirement exactly.
    - If :bash:`any` is False and the candidate does not match, but :bash:`empty` is True,
      attach an empty plugin from the registered default instead.
    - If :bash:`empty` is True and :bash:`any` is True but no named candidate was found,
      attach a bare empty class instance with only default class attributes.


.. warning::

    The dual options :bash:`subplug`/:bash:`preftree` should be avoided as much as possible as it means that
    your plugin is critically dependent on other external plugins.
    There are usually ways to avoid such implementation.


Dynamic requirements with ``set_requirements``
-----------------------------------------------

Before looping over the :bash:`requirements` dictionary, pyCIF calls the method
:bash:`set_requirements()` on the plugin instance.
Plugins can override this method to **modify their own** :bash:`requirements` **at runtime**,
for example to activate or skip a dependency based on a Yaml parameter already loaded:

.. code-block:: python

    def set_requirements(self):
        # Add a chemistry plugin only when the model has species
        if getattr(self, "use_chemistry", False):
            self.requirements["chemistry"] = {
                "name": "standard", "version": "std", "empty": True
            }

The base implementation of :bash:`set_requirements` does nothing.
This hook is intended for advanced plugin developers who need conditional dependencies.


Cheat sheet for checking plugins dependencies in yaml files
-----------------------------------------------------------

The following list can be obtained by running the following python commands.

.. code-block:: python

    from pycif.utils.classes.baseclass import Plugin
    Plugin.print_registered(print_rst=True, print_requirement=True)

.. toctree::
    :hidden:

    available

.. include:: available.rst