elias 0.3.3

ELIAS experiment library for facilitating machine learning projects

Experiment Library and Setup (ELIAS)

1. Main functionalities

Config class

Intuitive dataclass extension that addresses issues commonly encountered in research projects. A Config has the following features:

• Easily persistable to a file (JSON or YAML)
• Stored configs are human-readable and editable
• Special support for storing enums and inheritance structures
• Backward compatibility to allow loading older persisted configs

TODO: Overview image of benefits of using Config over regular dataclass

Philosophy for stored config file types:

• Everything should be human-readable
• JSON for configs that may be viewed often (statistics, preprocessing/training configs, evaluation results)
• YAML for configs that have to be edited (run specifications)

Experiment Workflow Utilities

To speed up the experimentation process with utility classes the elias library assumes the following workflow:

Stage	Input	Output	Utility
Data Preprocessing	Processing Configuration Raw Data	Preprocessed Data (.p, .p.gz, .json, .npy, ...) Data Statistics (stats.json) Preprocessing Config (config.json)	DataFolder -> DataManager
Training/Fitting	Preprocessed Data Hyperparameters	Model checkpoints Hyperparameter configs	ModelManager -> RunManager
Evaluation	Trained model Evaluation Config	Evaluation Config Evaluation Results	RunManager -> EvaluationManager
Manual Analysis	Any model/data	Plots, statistics, images	AnalysisFolder -> AnalysisManager