Merge your config files and set parameters from the command line in a simple way.
Project description
CLI Config
Lightweight library that provides routines to merge your configs (optionally nested) and set parameters from command line. It also prevents you from adding new parameters if not desired. Finally, it provides helper routines to manage flatten dicts, unflatten (= nested) dicts or a mix of both, save config, load, display, etc.
Documentation :memo: here
Installation
In a new virtual environment, simply install the package with:
pip install cliconfig
Quick start
First create a default config that can be split in multiple files that will be merged
(from left to right in make_config
function). There is no limit of depth for the
configuration parameters.
--- # default1.yaml
param1: 1
param2: 0
letters:
letter1: a
letter2: b
--- # default2.yaml
param1: 1
param2: 2 # will override param2 from default1.yaml
letters.letter3: c # add a new parameter
Now you can set up your program to use the config:
# main.py
from cliconfig import make_config, show_config
config = make_config('default1.yaml', 'default2.yaml')
show_config(config)
Then add one or multiple additional config files that will override the default values.
--- # first.yaml
letters:
letter3: C
--- # second.yaml
param1: -1
letters.letter1: A
The additional config files cannot add new parameters that are not in default configs. It is intended to prevent typos in the config files that would not be detected. It also improves the readability of the config files and the retro-compatibility.
Now you can launch the program with additional configurations and parameters.
The additional configurations are indicated with --config
(separate with comma,
without space) and the parameters with --<param_name>
(use dot for nested configs).
The default configuration will be merged with the additional configurations
(from left to right), then the parameters will be set.
python main.py --config first.yaml,second.yaml --param2=-2 --letters.letter2='B'
Will show:
[CONFIG] Merge 2 default configs, 2 additional configs and 2 CLI parameter(s).
Config:
param1: -1
param2: -2
letters:
letter1: A
letter2: B
letter3: C
Note that the configurations are native python dicts.
Edge cases
Be careful, tuples and sets are not supported by YAML and cannot be used in configs. Use lists instead if possible
None
is not recognized as a None object by YAML but as a string, you may use null
or Null
instead if you want to
set a None object.
Dicts are considered as sub-configs and so you may not be able to change the keys if
allow_new_keys=False
(default). If you want to modify or add dict keys, you should
enclose it in a list.
For instance:
--- default.yaml
logging:
metrics: ['train loss', 'val loss']
styles: [{'train loss': 'red', 'val loss': 'blue'}]
--- experiment.yaml
logging:
metrics: ['train loss', 'val loss', 'val acc']
styles: [{'train loss': 'red', 'val loss': 'blue', 'val acc': 'cyan'}]
Manipulate configs
You are encouraged to use the routines provided by cliconfig
to manipulate configs
and even create your own config builder to replace make_config
. You can use:
merge_config
andmerge_config_file
to merge your configsparse_cli
to parse CLI arguments on config files and additional parametersflat_config
andunflat_config
to flatten and unflatten your configsclean_pre_flat
to clean conflicting keys before flattening
You can also save, load and display configs with save_config
,
load_config
and show_config
functions.
Note that theses functions are aimed to be use with configs but can be used with any python dict.
How to contribute
For development, install the package dynamically and dev requirements with:
pip install -e .
pip install -r requirements-dev.txt
Everyone can contribute to CLI Config, and we value everyone’s contributions. Please see our contributing guidelines for more information 🤗
Todo
To do:
- log warning when
None
is used in CLI (considered as string a by PyYAML)
Done:
- add
merge_config_file
to merge from path (=merge_config
that includes config loading) - add
clean_pre_flat
to solve conflicting flatten and unflatten parameters before flattening the config - avoid changing keys order in merge_config
License
Copyright (C) 2023 Valentin Goldité
This program is free software: you can redistribute it and/or modify it under the terms of the MIT License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
This project is free to use for COMMERCIAL USE, MODIFICATION, DISTRIBUTION and PRIVATE USE as long as the original license is include as well as this copy right notice at the top of the modified files.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cliconfig-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bb3d14902ef231de5412507e536e775301e786924e6ef6be2d3a71b56bbd76f |
|
MD5 | 9f44fb6e143e306e8c1772b4dbef57d4 |
|
BLAKE2b-256 | 68b8faf0fc75140319c6e9928e1fef5aaa6bcc892b7b239c0fb8f88fe3627444 |