ADR-0006: Single Plugin base class + type/subtype registry as the extensibility mechanism

ADR-0006: Single Plugin base class + type/subtype registry as the extensibility mechanism#

Field

Value

Kind

ADR

Status

Accepted

Decided

historical

Deciders

Antoine Berchet

Supersedes

—

Superseded by

—

Context#

CIF needs many interchangeable implementations along many independent axes — transport models, observation operators, minimizers, domains, control vectors, HPC platforms, transforms, inversion modes, and more (16 recognised plugin types today: chemistry, controlvect, datavect, datastream, domain, measurements, minimizer, mode, model, obsoperator, obsparser, obsvect, platform, simulator, transform, setup). Every one of them needed a common way to be declared in the user’s YAML configuration, discovered on disk, dynamically imported, instantiated, validated against its accepted arguments, and wired to the plugins it depends on — without inventing a bespoke loading mechanism per subsystem.

CIF’s contributors are largely atmospheric scientists, not software engineers, so the mechanism also had to keep the barrier to adding a new plugin low: a contributor should be able to add a new model, obs operator, or transform by writing the required routines and declaring their arguments inside a plugin directory, without needing to understand Python class syntax, inheritance, or how CIF’s internals wire plugins together — CIF builds and interfaces the instance dynamically from that directory’s conventions.

See Documentation for the plugin catalogue this produces, and pycif/utils/classes/baseclass.py for the implementation.

Decision#

We implement extensibility as a single Plugin base class with:

  • a class-level plugin_types registry mapping each recognised type name to its module path and class name, plus plugin_subtypes for nested categories (e.g. datastream → meteo / flux / background / field; transform → basic / complex / system);

  • a YAML plugin: {name, version, type, subtype} declaration every plugin instance uses to identify itself;

  • global registries (registered, loaded_instances, reference_instances, subreference_instances) that dynamically importlib-load and cache plugin modules/instances;

  • an input_arguments convention each plugin module declares for its own accepted YAML keys, checked against at load time (unauthorized_arguments).

Every plugin category subclasses Plugin (e.g. Platform(Plugin), Model(Plugin), …) rather than defining its own independent loading mechanism.

Consequences#

  • Adding a new plugin category means one entry in plugin_types / plugin_subtypes plus one subclass — loading, registry, and YAML-wiring are inherited for free.

  • Adding a new plugin within an existing category means writing a module with the expected _name / _version / input_arguments conventions under pycif/plugins/<type>/<name>/ — there is no registration code to write by hand.

  • This base class is the “plugin interface contract” referred to elsewhere in the contribution rules (a new plugin conforming to an existing interface needs no RFC; changing the interface itself does) — see docs/decisions/README.md.

  • It also underpins config validation and JSON-Schema generation (ADR-0008): a plugin’s declared input_arguments is the single source of truth for what it accepts. Any documentation describing plugin arguments that isn’t generated from input_arguments can silently drift from what the code actually validates.