compile

The compile config group determines whether to activate torch.compile upon the model. This may speed up model training but has limitations (it requires an initial optimization routine when the model starts training) that leads to overhead which may not be regained by (potentially lacking) training speed improvements. Furthermore there have been issues with the continue functionality. Thus it is not enabled by default.

enable

default: False

• enables torch.compile on the model returned by base schedulers create_model()

• some recommendations can be found here https://lightning.ai/docs/fabric/2.4.0/advanced/compile.html

kwargs

• the kwargs passed to https://pytorch.org/docs/main/generated/torch.compile.html#torch-compile

mode

default: reduce-overhead