mml.core.scripts.schedulers.preprocess_scheduler

class PreprocessScheduler

AbstractBaseScheduler implementation for the process of preprocessing data. Includes the following subroutines: - preprocess

__init__(cfg: DictConfig)

after_preparation_hook()

before_finishing_hook()

create_routine()

This scheduler implements one subroutine, which preprocesses a task’s data.

Returns:

None

preprocess_task(task_name: str)

preprocess_and_store(index: int, ds: TaskDataset, target_base: Path, transform: AlbumentationsAugmentationModule) → Dict[str, bool]

Function to preprocess and store a single data tuple of a dataset.

Parameters:

• ds – dataset

• target_base – root path to store preprocessed data

• index – index of data tuple to be preprocessed

• transform – an albumentations transform module to be applied on loaded data

Returns:

dict with modality keys and boolean indicating whether a file already existed beforehand