Tuning

Via pytorch-lightning mml offers automatic tuning of the initial learning rate as well as the batch size. To activate this behaviour simply refer tune.lr=true and tune.bs=true respectively. Note that learning rate tuning is activated by default!

To provide more granular access to lightnings tuning options you may use the tune.lr_kwargs (or tune.bs_kwargs) e.g.

!mml train tasks=fake tune.lr=true +tune.lr_kwargs.mode=linear +tune.lr_kwargs.early_stop_threshold=2.0 trainer.max_epochs=1