Opinionated config handling for BCG Gamma projects
Project description
Gamma Config
An opinionated way of setting up configuration for data science projects.
Overview
Gamma config provides a standard and flexible way of implementing application configuration primarily based on YAML files. It promotes best practices by:
- simplify separation of code and config data;
- breaking large, complex configuration into multiple files;
- providing a way to write environment-aware config files;
- facilitating security best-practices, like proper secrets management;
- providing a immutable central global object storing all contextual data.
Another benefit of a standard config mechanism is allowing Gamma extension libraries to provide configuration in a common way.
Features
- Configuration expressed as a set of YAML files (by defaul) inside a
config
folder in the root of the project. - Multiple YAML files merged following simple rules. Simple file ordering convention using two digit prefixes.
- Builtin support for environment specific parameters (production, development, etc.)
- Support for
.env
files viapython-dotenv
, including.local.env
that can be added to.gitignore
- Dynamic evaluation via YAML tags. For instance, the
!ref
tag allow you to reference other parameters in any config file. - Custom tag support via simple and cool multiple dispatch mechanism.
- Round-trip dump of config back into YAML. Support for hiding sensitive data on dump.
- Simplified key access via dot (
.
). Eg. forconfig: {foo: {bar: 100}}
, this is True:config.foo.bar == 100
Full documentation
Link: https://github.gamma.bcg.com/pages/BCG/gamma-config/
Getting started
Currently the recommended way of installing gamma-config
is downloading the tar.gz
asset file in the Releases
and install it using pip
.
pip install ./gamma-config-<release>.tar.gz
To use it in your project, you can place the tar.gz files in a vendor
folder and
reference it either via pip's -f
flag or by setting PIP_FIND_LINKS
env var:
# requirements.txt:
gamma-config
# then explicity set -f (--find-links) flag
pip install -r requirements.txt -f ./vendor
Poetry currently does not support the equivalent of --find-links
so you may need to
explicity add the path/version to pyproject.toml
[tool.poetry.dependencies]
gamma-config = { path = "./vendor/gamma-config-<release>.tar.gz" }
This technique is called "vendoring" the dependency. For client work this also provides the library source should they wish to modify it later.
In most cases, you'll want to use the !j2
tag to interpolate values using Jinja2.
This requires manually installing the jinja2
package or using the jinja2
extras.
# when using pip
pip install gamma-config[jinja2]
Basic Usage
The package comes with "scaffolding" to help you get started. In your project folder:
python -m gamma.config.scaffold
Remove the sample files, then create yourself a config/20-myconfig.yaml
file
with the contents:
foo: 1
user: !env USER
To access the config from within your Python program:
import os
from gamma.config import get_config
def run():
# it's safe and efficient to call this multiple times
config = get_config()
# get static value using the dict keys or attribute access
assert config["foo"] == 1
assert config.foo == 1
# get dynamic variables
assert config["user"] == os.getenv("USER")
assert config.user == os.getenv("USER")
Most of the magic happen via tags. Look at the documentation for info on the built-in tags available.
Copyright
Copyright 2021 Boston Consulting Group, all rights reserved.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gamma_config-0.4.18-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9e50d3e2c2a0d165aac57e67be487b23db58c9e76b48c2fdbfa9f1735a78f14 |
|
MD5 | d820fa5a7f13e5d434f217f27643b9b4 |
|
BLAKE2b-256 | 013fb7b1751a491c8e701df97bfb947309fd796dc220296aade14d1df1d96826 |