remove

The remove config group determines the MMLFileManager’s behaviour at the end of an experiment. During the finish_exp() routine of the experiment’s scheduler the remove_intermediates() method of the MMLFileManager is called and triggers the deletion of created files according to the settings of this config group. This is useful to get rid of intermediates or general results that are not used any more and may use up the disk capacity.

The path assignment system of the MMLFileManager determines a key for each kind of path assignment. This key is used within this config to trigger deletion of files. The default path assignments (and according keys) are defined in DEFAULT_ASSIGNMENTS. Here are some examples:

  • models: the ModelStorage for each trained model

  • parameters: the ModelCheckpoint for a model

  • pipeline: the PipelineCfg for a training process

  • predictions: predictions of a model on a task, dictionary stored with torch.save

By default (remove=none) no data is deleted (except for inherently temporary data, see remove_intermediates()). The alternative config file remove=all will take care of most of the default assignments. Any additional key can be added via prepedning a + as in +remove.MY_KEY=true, which might be necessary for plugin provided path assignments.

all

backup
default: True

  • backup files

blueprint
default: True

  • training pipeline blueprints

models
default: True

  • model storage

parameters
default: True

  • model weights

pipeline
default: True

  • training pipeline

predictions
default: True

  • model predictions

sample_grid
default: True

  • grid of example images as produced by the info mode

img_examples
default: True

  • example images as produced by the info mode