augmentations

The augmentations config group determines the image augmentations applied by mml during the loading of train data (on top of any preprocessing). Currently there are three backbone libraries: albumentations, torchvision and kornia. While albumentations works on CPU only, kornia is implemented GPU only in mml. Torchvision is flexible in being applied on both device types. Multiple pipeline config files are shipped with mml as examples:

• augmentations=base_rand - basic RandAugment

• augmentations=kornia - a default GPU transformation pipeline with kornia

• augmentations=load_imagenet_aa - a automatically learned augmentation pipeline (by autoalbument)

• augmentations=v2 - an example with torchvision transforms

Note that it is possible to combine CPU and GPU transforms. mml takes care of formatting, scaling, tensorizing, etc. so the config pipelines may fully focus on the relevant image transformations. Complex pipelines are best created as .yaml files and called via file name. You may use the Config copy functionality to create your own set of plain config files to modify. The relevant entries are documented below:

normalization

default: imagenet

• determines the normalization strategy for images

• ‘imagenet’: the imagenet mean and std of RGB channels are used, this is recommended if using imagenet pretrained models

• ‘task’: use mean and std of RGB channels for each task, this is recommended if training from scratch

• ‘pretraining’: try to extract pretraining normalization from the model, this may be less reliable than ‘imagenet’ option

• ‘null’: do not normalise, e.g. to be used if images shall only be plotted but not fed through a model

cpu

backend

default: albumentations

• backend for cpu augmentations, currently supports albumentations (https://albumentations.ai/) and torchvision (https://pytorch.org/vision/stable/transforms.html#start-here)

• the backend library determines the available transforms, which are listed in the cpu.pipeline config

pipeline

default: [{name:RandomCrop,height:224,width:224},{name:ShiftScaleRotate,shift_limit:0.05,scale_limit:0.05,rotate_limit:15},{name:HorizontalFlip,p:0.5},{name:RandomBrightnessContrast,p:0.5},{name:CoarseDropout,p:0.5}]

• the actual augmentation pipeline, listed as a sequence of dictionaries with a ‘name’ each and optional kwargs

• available albumentations transforms are: https://albumentations.ai/docs/api_reference/full_reference/

• available torchvision transforms are: https://pytorch.org/vision/stable/transforms.html#v2-api-reference-recommended

• currently nesting / containers are not supported!

• formatting transformations (e.g. ToFloat / Normalize / ToTensor) are beeing taken care of mml automatically

• albumentations backend two custom added transform named ‘ImageNetAA’ (auto augment generated pipeline, no kwargs) and ‘RandAugment’ (see get_rand_augment())

gpu

default: {}

• gpu augmentations are structured similarly to cpu augmentations and require backend and pipeline specification

• empty dictionary means no augmentations

• supported backends are kornia and torchvision

• available kornia transforms: https://kornia.readthedocs.io/en/latest/augmentation.html

• nesting and containers are not supported either