##################################################################
Automatic resubmission of jobs
##################################################################

This option is primarily intended for CCRT users. Since batch job walltime is limited to 3 days, it may be necessary to relaunch long inversions automatically.
To enable this, set two optional arguments in :bash:`obsoperator` as described below.
The inversion will stop automatically after :bash:`autokill_time` and a new job will be submitted.
Set a sufficient margin between :bash:`autokill_time` and the cluster walltime, because the kill/resubmit process occurs at the end of the current transform loop.
This margin must be large enough to ensure all running transforms finish before the job is killed.
To enable automatic resubmission:

1) Modify the YAML to enable resubmission:

   - in :bash:`obsoperator`, set option :bash:`autokill_time` to the duration after which the job is resubmitted.
   - in :bash:`obsoperator`, set option :bash:`max_resubmissions`.

.. code-block:: yaml

    obsoperator:
      plugin:
        name: standard
        version: std
        type: obsoperator
      autokill_time: 71H
      max_resubmissions: 5
      
2) Depending on your CIF configuration at CCRT, you may need to add a Python userbase path via :bash:`python_userbase` and a virtual environment path via :bash:`python_venv`.
3) A new job will be submitted when :bash:`autokill_time` is reached, enabling inversions that span periods longer than 3 days.