Skip to main content

Very opinionated configuration loading package for Python projects

Project description

Confuk

This is yet another package for managing configuration files in Python projects.

It exposes one function that lets you provide a path to a TOML/YAML/JSON configuration file. It parses the config file into a dictionary by default. If a config class is provided when parsing, the class instance will be created using a dictionary of keyword arguments coming from the original TOML/YAML/JSON file.

In human words: I made this package so that I don't have to explicilty load, parse and return a class instance every single time I have something to do with a configuration file:

from confuk import parse_config
from pathlib import Path
from somewhere import ConfigClass

cfg_dict = parse_config(Path("some.toml"))  # returns a dictionary
cfg_obj = parse_config(Path("some.toml"), ConfigClass)  # returns an instance of `ConfigClass`

[!tip] confuk also supports a number of output configuration styles out-of-the-box, including omegaconf, Pydantic and EasyDict.

Installation

pip install confuk

Or:

poetry add confuk

Building from source

You can build the package using Poetry:

  1. Clone this repo.
  2. Run poetry build.
  3. Grab the installable wheel from the dist folder and install it with pip or add the package as a local dependency of another project.

Special features

Config output formats

EasyDict

If you really hate referring to dictionary keys and you do not intend to create a custom configuration class for your config, you can parse the file to an EasyDict:

cfg_edict = parse_config(Path("some.toml"), "attr")

Now, if the key something exists in the configuration file, you can simply refer to it using an attribute:

cfg_edict.something

OmegaConf

OmegaConf is one of the most complete configuration systems for Python applications. If you want to leverage its features while still working with confuk as a front-end, you can simply parse the configuration into an instance of omegaconf.DictConfig by doing the following:

cfg = parse_config(Path("some.toml"), "omega")

Pydantic

If you're a fan of Pydantic with custom config classes for automatic validation, just use any class that inherits from BaseModel:

from confuk import parse_config
from pathlib import Path
from pydantic import BaseModel

class Metrics(BaseModel):
    psnr: float
    ssim: float


cfg_dict = parse_config(Path("some.toml"), Metrics)  # returns a dictionary

All currently supported formats

Format cfg_class argument
dict "d" / None
EasyDict "ed" / "edict" / "attr"
OmegaConf "o" / "omega" / "omegaconf"
pydantic BaseModel class
custom any class supporting **kwargs in the constructor

Imports

Because keeping hundreds of config files can become tedious, especially when there is shared values between them, you might want to consider using the imports functionality.

Say you have a TOML file from which you want to inherit values:

[something]
value = 1
another_value = 2

[something_else]
value = 3

You can "import" it using a preamble:

[pre]
imports = [
    "${this_dir}/test_imported.toml",
]

[something]
value = 69

[!note] Older versions of confuk used the $this_dir syntax instead. This will be supported going into the future but it won't work with variable interpolation (expect it to only work for the special interpolation markers such as $this_dir and $cwd).

This is equivalent to specifying a config like:

[something]
value = 69
another_value = 2

[something_else]
value = 3

Note that you can use several special interpolation markers to specify paths in the import section:

  • ${this_dir} -> points to a directory relative to the configuration file that contains the import section
  • ${cwd} -> points to the current working directory
  • ${this_filename} -> config filename (with extension)
  • ${this_filename_stem} -> filename without the extension (stem)

[!warning] The preamble will be removed after it's processed. It's there only to control how confuk should process the loaded configuration files and it's dropped afterwards. Do not put any meaningful configuration into your preamble, except for confuk's control elements.

What about inheriting selected values?

Unsupported. And I do not plan to add support for cherrypicking values from other configs. It makes things way messier in my opinion, as it becomes way harder to reason about the flow of variables.

What about variable interpolation?

This is supported with the syntax that OmegaConf uses, e.g. path = "${some.root.path}/file.txt" will pick up the path variable from some.root config section. The interpolation markers that I mentioned in the Imports section should also work anywhere else within the config, so you can use your ${this_filename_stem} to refer to config names within the config itself. One use-case is when you want to have subdirectories in a results directory, where you would silo away the results from different configs:

results_dir = "results/${this_filename_stem}"

Assumming that you have 3 configs for your experiments: ex1, ex2 and ex3, you could instead put results_dir in a parent config to all those:

# to_import.toml:
results_dir = "results/${this_filename_stem}"

# ex1
[pre]
imports = ["${this_dir}/to_import.toml"]
a_variable_that_diverges_across_configs = 69

# ex2
[pre]
imports = ["${this_dir}/to_import.toml"]
a_variable_that_diverges_across_configs = 420

# ex3
[pre]
imports = ["${this_dir}/to_import.toml"]
a_variable_that_diverges_across_configs = 42

[!note] We are using omegaconf for all other interpolation tasks under the hood since they already have a great parser for this and there's no use duplicating work.

What about deeply nested configs?

If you like the deeply nested folder-file structure for your configs then Hydra might be more for you. I've used it before and it's very good but I personally find the design choice of creating directory structures for configs quite tedious.

confuk strives to be flatter: you import another config file in the preamble section and you have a choice of what to override. This makes it more comfortable to use when you have one default.toml config file for something and then create a bunch of configurations overriding certain values. This is useful for experiments in the AI/ML space, where I'm spending most of my time now.

You are of course free to structure your files as you please but don't expect a feature similar to Hydra's defaults in confuk – I do indeed use Hydra for applications which require such a system!

Command-line overrides

One of the most fantastic features I've found when using Hydra was the ability to override values from the config file on the command line. This is convenient when you want to quickly test some changes to your configuration without going through the trouble of creating a new config file.

So I concluded it would be fun to implement it in confuk in a similar fashion. Here's how it works:

import confuk


@confuk.main(config=Path(__file__).parent / "test.toml", config_format="o", verbose=False)
def main(cfg, *args):
    console = Console()
    console.print(cfg)
    return cfg

This decorator behaves similarly to @hydra.main decorator and it creates a minimal argument parser for your application entrypoint under the hood.

Now, when running the app, you can specify any value overrides on the command line. For example if your config looks like this:

[my]
mother = 1

[your.dad]
father = 1

And you run your CLI app with the argument your.dad.father=3, you will override the pertinent value from 1 to 3.

[!tip] The underlying argument parser also contains a --config option. You can use it to switch to a different config path on the command line, without a need to rely on the default one that has been set in the decorator.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

confuk-0.4.1.tar.gz (9.3 kB view details)

Uploaded Source

Built Distribution

confuk-0.4.1-py3-none-any.whl (8.6 kB view details)

Uploaded Python 3

File details

Details for the file confuk-0.4.1.tar.gz.

File metadata

  • Download URL: confuk-0.4.1.tar.gz
  • Upload date:
  • Size: 9.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.13.0

File hashes

Hashes for confuk-0.4.1.tar.gz
Algorithm Hash digest
SHA256 6dd3f16e2f96cb2ad477bc91f4e057136b294268aa8af5a31c075a367bab47f8
MD5 1e313597b36daeb76d0beb56ee71e1e2
BLAKE2b-256 5e2157fb1616987110336d2400b0be190d8da70afdf00d69df7e47401af84652

See more details on using hashes here.

File details

Details for the file confuk-0.4.1-py3-none-any.whl.

File metadata

  • Download URL: confuk-0.4.1-py3-none-any.whl
  • Upload date:
  • Size: 8.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.13.0

File hashes

Hashes for confuk-0.4.1-py3-none-any.whl
Algorithm Hash digest
SHA256 9de8420dce107b6152ac3360f3a191de5d7f9d504aacd602fa0287d85a17555c
MD5 7425cc524825b30ed06c890da15e6d75
BLAKE2b-256 0c49aa9952d7ad1f20e394a955ec2e0693bf385c94126dc81dfb5db07e2b48b5

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page