hpo
The hpo config group controls hyperparameter optimization searches. It is triggered by the --multirun flag when
invoking an mml experiment (see hydra’s documentation).
If using an:class:~mml.interactive.planning.MMLJobDescription there is the multirun boolean flag.
use_best_params
default
- hpo
this config uses the optuna sweeper plugin of hydra
the considered hyperparameters are given by the search_space config
will sequentially run hpo.trials many experiments and sample hps according to hpo/sampler
default sampler is tpe (https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html)
other samplers are
hpo/sampler=random(https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.RandomSampler.html) andhpo/sampler=grid(https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.GridSampler.html)example:
mml train proj=test search_space=example hpo/sampler=grid --multirunmore details can be found in hydra’s optuna sweeper documentation
- storage
- default: null
the optuna storage to use, if null only a temporary in-memory storage will be created
see mml-sql for details on how to set up permanent storage
- direction
- default: minimize
direction of optimization, either minimize or maximize
must be set according to the kind of return value by the respective scheduler
- pruning
- default: False
currently unused, intended for future pruning of hopeless runs
- trials
- default: 100
number of trials, each trial is one mml call with (potentially) different sampled hyperparameters
grid
- storage
this config corresponds to the default multirun variant of hydra
use colons to provide multiple values via CLI (the search_space config is not used!)
will sequentially run one exp for each possible combination of config values (using the colon syntax)
example:
mml train tasks=fake proj=test hpo=grid optimizer.lr=0.001,0.01 --multirunmore details can be found in hydra’s multi-run documentation