Yaml configuration

Yaml configuration#

pyCIF can be entirely set up with a Yaml file such as in the example below. The Yaml configuration file structure is used to define, initialize and run the building blocks of pyCIF: Plugins.

General info about Yaml#

The basic idea of Yaml syntax is that you can easily define a tree structure using ‘:’ and indentation, which would be automatically interpreted by Python.

Please note that Yaml is flexible with the keys it accepts for booleans. The following keys will be automatically accepted as True or False:

y / Y / yes / Yes / YES / n / N / no / No / true / True / TRUE / false / False / FALSE / on / On / ON / off / Off / OFF

Warning

Please note that NO will not be interpreted as False, contrary to expected behaviour.

This is to avoid mis-interpretation with NO for Nitrous Oxyde.

For instance, the following Yaml syntax:

root:
   branch1: grub
   branch2: [0, 1]
   branch3:
      subbranch0: some text
      subbranch1: 2004-01-01
      subbranch2: T

will be interpreted into the following Python dictionary:

{
'root':
   'branch1': 'grub',
   'branch2': [0,1],
   'branch3':
      'subbranch0': 'some text',
      'subbranch1': datetime.datetime(2004, 01, 01),
      'subbranch2': True
}

which is itself loaded into a pyCIF plugin automatically integrated with other plugins and dependencies.

Further examples about the Yaml syntax can be found here.

The initialized dictionary is then loaded as Plugins for later use by pyCIF.

Yaml and pyCIF#

There is a minimal number of arguments mandatory to run a pyCIF simulations. They are:

  1. verbose: the verbose level of the simulations. pyCIF uses the logging python package. Log instances are determined by the commands info, debug and warning

    Verbose levels corresponds to levels in logging:

    • verbose = 0: debugging verbose. Includes the path to the python script were the log instance was called

    • verbose = 1: debugging verbose, without extra information on the script path

    • verbose = 2: only main information are displayed

    • verbose > 2: only warnings are displayed

  2. workdir: directory were the pyCIF computations will be saved

  3. logfile: name of the logfile

  4. datei and datef: the start and end dates of the simulations

Note

Please consult the page here for details on how paths are handled in pyCIF.

The most basic Yaml file is thus:

1#####################
2# pyCIF config file #
3#####################
4
5verbose: 2          # 0=debug (with path), 1=debug, 2=info, >2=warnings only
6logfile: pycif.log
7workdir: /path/to/your/workdir/
8datei: 2010-01-01
9datef: 2010-01-05

Diagnostic and profiling options#

Several diagnostic options can be set either on the command line or directly in the YAML file. The command-line flag always takes priority when both are present. Setting them in the YAML is especially useful for batch jobs (SLURM, PBS) where adding extra CLI arguments to the submission script is inconvenient.

YAML key

Type

Description

monitor_memory

bool

Log peak and current RSS after every obs-operator transform step. Equivalent to --monitor_memory.

profiling

string

Write a pyinstrument HTML profile report to this path (relative to workdir, or absolute) when the run finishes. Equivalent to --profiling. Requires pyinstrument (pip install pyinstrument).

tracking

string

E-mail address to receive a structured crash report on uncaught exception. Equivalent to --tracking. SMTP host/port are controlled by the PYCIF_ERROR_SMTP_HOST / PYCIF_ERROR_SMTP_PORT environment variables (default: localhost:25).

Example combining all three:

verbose: 2
logfile: pycif.log
workdir: /scratch/user/run01/
datei: 2022-01-01
datef: 2022-01-31

monitor_memory: true
profiling: profile.html          # written to workdir/profile.html
tracking: oncall@lab.example.fr  # emailed on crash

For full details on each option — including SMTP configuration, report format, and interaction with --debug — see General options.