ml-confs 0.0.1

A utility to handle configurations for machine learning pipelines

ML configurations

A small, highly opinionated python tool to handle configurations for machine learning pipelines. The library is designed to load configurations from both json and yaml files, as well as from standard python dictionaries.

Design rules

The configurations, once loaded are frozen. Each configuration file can contain only int, float, str, bool and None fields, as well as homogeneous lists of one of the same types. That's all. No nested structures are allowed.

Installation

ML configurations can be installed directly from git by running

pip install git+https://github.com/Pietronvll/ml_confs.git

Basic usage

A valid ml_confs configuration file configs.yml in YAML is:

int_field: 1
float_field: 1.0
str_field: 'string'
bool_field: true
none_field: null
list_field: [1, 2, 3]

To load it we just use:

import ml_confs as mlcfg

#Loading configs
configs = mlcfg.from_file('configs.yml')

#Accessing configs with dot notation
print(configs.int_field) # >>> 1

#Additionally, one can use the ** notation to unpack the configurations
def foo(**kwargs):
    # Do stuff...
foo(**configs)


#Saving configs to json format
mlcfg.to_file('json_configs_copy.json') #Will create a .json file 

One can also pretty print a loaded configuration with ml_confs.pprint, which in the previous example would output:

┏━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Key         ┃ Value     ┃ Type      ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━┩
│ int_field   │ 1         │ int       │
│ float_field │ 1.0       │ float     │
│ str_field   │ string    │ str       │
│ bool_field  │ True      │ bool      │
│ none_field  │ None      │ NoneType  │
│ list_field  │ [1, 2, 3] │ list[int] │
└─────────────┴───────────┴───────────┘

API reference

function from_json

from_json(path: PathLike, flax_dataclass: bool = False)

Load configurations from a JSON file.

Args:

• path (os.PathLike): Configuration file path.
• flax_dataclass (bool, optional): Returns a flax compatible object. Uses flax.struct.dataclass. Defaults to False.

Returns:

• Configs: Instance of the loaded configurations.

---

function from_yaml

from_yaml(path: PathLike, flax_dataclass: bool = False)

Load configurations from a YAML file.

Args:

• path (os.PathLike): Configuration file path.
• flax_dataclass (bool, optional): Returns a flax compatible object. Uses flax.struct.dataclass. Defaults to False.

Returns:

• Configs: Instance of the loaded configurations.

---

function from_dict

from_dict(storage: dict, flax_dataclass: bool = False)

Load configurations from a python dictionary.

Args:

• storage (dict): Configuration dictionary.
• flax_dataclass (bool, optional): Returns a flax compatible object. Uses flax.struct.dataclass. Defaults to False.

Returns:

• Configs: Instance of the loaded configurations.

---

function from_file

from_file(path: PathLike, flax_dataclass: bool = False)

Load configurations from a YAML/JSON file.

Args:

• path (os.PathLike): Configuration file path.
• flax_dataclass (bool, optional): Returns a flax compatible object. Uses flax.struct.dataclass. Defaults to False.

Returns:

• Configs: Instance of the loaded configurations.

---

function to_json

to_json(path: PathLike, configs: BaseConfigs)

Save configurations to a JSON file.

Args:

• path (os.PathLike): File path to save the configurations.
• configs (BaseConfigs): Instance of the configurations.

---

function to_yaml

to_yaml(path: PathLike, configs: BaseConfigs)

Save configurations to a YAML file.

Args:

• path (os.PathLike): File path to save the configurations.
• configs (BaseConfigs): Instance of the configurations.

---

function to_file

to_file(path: PathLike, configs: BaseConfigs)

Save configurations to a YAML/JSON file.

Args:

• path (os.PathLike): File path to save the configurations.
• configs (BaseConfigs): Instance of the configurations.

---

function to_dict

to_dict(configs: BaseConfigs) → dict

Export configurations to a python dictionary.

Args:

• configs (BaseConfigs): Instance of the configurations.

Returns:

• dict: A standard python dictionary containing the configurations.

---

function pprint

pprint(configs: BaseConfigs)

Pretty print configurations.

Args:

• configs (BaseConfigs): An instance of the configurations.

---

The API reference was automatically generated via lazydocs.