lr_scheduler

Provides the possibility to choose a learning rate scheduler. Should be compatible with any lr_scheduler``by ``torch` (see here for a list). The internal calls to the scheduler are dealt with by lightning. Most schedulers have a couple of hyperparameters, which are not all mapped to the yaml config files. You may add those from CLI via prepending a + (e.g. lr_scheduler=plateau +lr_scheduler.cooldown=3). Below you can find the config files, but adding more should be straightforward from the examples provided. By default no scheduler is used.

none

_target_
default: null

  • pseudo target to be caught within code and not to be instantiated

cosine

_target_
default: CosineAnnealingLR
T_max
default: ${trainer.max_epochs}

  • resolved by hydra

cosine_restart

_target_
default: CosineAnnealingWarmRestarts
T_0
default: 25

  • epochs until first restart

exponential

_target_
default: ExponentialLR
gamma
default: 0.99

  • multiplicative factor of learning rate decay

one_cycle

_target_
default: OneCycleLR

  • one cycle learning rate scheduler

  • see OneCycleLR

max_lr
default: 0.001

  • max learning rate

epochs
default: ${trainer.trainer.max_epochs}

  • number of epochs

  • inferred by hydra

plateau

_target_
default: ReduceLROnPlateau
mode
default: min

  • direction of improvement (min for loss)

factor
default: 0.1

  • reduction factor if triggered by plateau

patience
default: 5

  • number of epochs to wait for improvement before triggering a reduction

threshold
default: 0.0001

  • improvement threshold to determine if an improvement happened

step

_target_
default: StepLR

  • step wise learning rate scheduler

  • see StepLR

step_size
default: 25

  • epochs to wait

gamma
default: 0.1

  • multiplicative factor