ConfStack

Summary: A multi-layer configuration system for Pydantic models. Layers are applied in priority order (lowest to highest): (1) in-code defaults, (2) configuration file, (3) lower-dotted env vars, (4) upper-underscored env vars, (5) programmatic overrides, (6) CLI arguments. Required fields without a default are reported with a clear error listing every way to set them.

Layer	Priority	Name	Quick Example
1	Lowest	In-code Defaults	key: str = "value"
2		Configuration File	{"key": "value"}
3		Lowercase Dotted Env. Vars	app.key=value
4		Uppercase Underscored Env. Vars	APP_KEY=value
5		Programmatic Overrides	{"key.sub": "value"}
6	Highest	CLI Arguments (built-in)	--key value

Installation

pip install confstack

Requires Python 3.9+ and Pydantic 2.10+.

Quick Start

See src/confstack/example.py for the full model — it defines an AppCfg with 3-level nesting and required fields at multiple depths.

import confstack
from confstack.example import AppCfg

config = confstack.confstackify(AppCfg, "example_app")

Fields with no default are required — they must be supplied via config file, env vars, overrides, or CLI. If any are missing, confstackify raises a ConfstackError with explicit guidance:

ConfstackError: Required config fields not set:
  key_00  =>  set via: --key_00 CLI flag, EXAMPLE_APP_KEY_00 env, or example_app.key_00 env
  key_02.subkey_00  =>  set via: --key_02.subkey_00 CLI flag, EXAMPLE_APP_KEY_02_SUBKEY_00 env, etc.
  key_03.subkey_01.subsubkey_02  =>  set via: --key_03.subkey_01.subsubkey_02 CLI flag, etc.

Supply them and it works:

EXAMPLE_APP_KEY_00=1 \
EXAMPLE_APP_KEY_02_SUBKEY_00=2 \
EXAMPLE_APP_KEY_03_SUBKEY_01_SUBSUBKEY_02=3 \
python -m confstack.example

The example script uses confstack.diff to show which values were overridden vs. defaults:

@@ -1,20 +1,20 @@
 {
-  "key_00": "[#!UNSET::layer_01_value_00]",
+  "key_00": "1",
   "key_01": "layer_01_value_01",
   "key_02": {
-    "subkey_00": "[#!UNSET::layer_01_value_02_01]",
+    "subkey_00": "2",
     "subkey_01": "layer_01_value_02_01",
     "subkey_02": "layer_01_value_02_02",
     "subkey_03": "layer_01_value_02_03"
   },
   "key_03": {
     "subkey_00": {
       "subsubkey_00": "layer_01_value_03_00_00"
     },
     "subkey_01": {
       "subsubkey_00": "layer_01_value_03_01_00",
       "subsubkey_01": "layer_01_value_03_01_01",
-      "subsubkey_02": "[#!UNSET::layer_01_value_03_01_02]"
+      "subsubkey_02": "3"
     }
   }
 }

Layer Details

Layer 1 : In-code Defaults

Any Pydantic model works. Nested models define sections. Required fields (no = "...") must be supplied from higher layers — ConfStack will tell you exactly how if they are missing.

import pydantic as pdt
import typing as tp


class Config(pdt.BaseModel):
    key_00: str                                  # required
    key_01: str = "layer_01_value_01"

    class Key02(pdt.BaseModel):
        subkey_00: str                           # required
        subkey_01: str = "layer_01_value_02_01"
        subkey_02: str = "layer_01_value_02_02"
        subkey_03: str = "layer_01_value_02_03"

    key_02: tp.Optional[Key02] = pdt.Field(default_factory=Key02)

    class Key03(pdt.BaseModel):
        class Subkey00(pdt.BaseModel):
            subsubkey_00: str = "layer_01_value_03_00_00"

        subkey_00: Subkey00 = pdt.Field(default_factory=Subkey00)

        class Subkey01(pdt.BaseModel):
            subsubkey_00: str = "layer_01_value_03_01_00"
            subsubkey_01: str = "layer_01_value_03_01_01"
            subsubkey_02: str                    # required

        subkey_01: Subkey01 = pdt.Field(default_factory=Subkey01)

    key_03: tp.Optional[Key03] = pdt.Field(default_factory=Key03)

Layer 2 : Configuration File

Default path: ~/.config/{app_name}/config.json. Nested JSON maps directly to nested model fields.

{
  "key_00": "from_config",
  "key_02": {
    "subkey_00": "from_config",
    "subkey_01": "from_config"
  }
}

Pass a custom path:

confstack.confstackify(Config, "myapp", config_file="/path/to/custom.json")

Layer 3 : Lowercase Dotted Environment Variables

env \
  app_name.key_00="from_lower_env" \
  app_name.key_02.subkey_00="from_lower_env" \
  python main.py

Works well with Docker Compose:

services:
  app:
    environment:
      app_name.key_00: from_lower_env
      app_name.key_02.subkey_00: from_lower_env

Layer 4 : Uppercase Underscored Environment Variables

Uppercase env vars override lowercase ones for the same path.

APP_NAME_KEY_00="from_upper_env" \
APP_NAME_KEY_02_SUBKEY_00="from_upper_env" \
  python main.py

services:
  app:
    environment:
      APP_NAME_KEY_00: from_upper_env
      APP_NAME_KEY_02_SUBKEY_00: from_upper_env

Layer 5 : Programmatic Overrides

Accepts nested dicts or flat .-separated keys:

# Nested dict
confstack.confstackify(Config, "myapp", overrides={"key_02": {"subkey_00": "override"}})

# Flat dotted keys
confstack.confstackify(Config, "myapp", overrides={"key_02.subkey_00": "override"})

Layer 6 : CLI Arguments

Enable built-in CLI parsing with parse_cli_args=True:

config = confstack.confstackify(Config, "myapp", parse_cli_args=True)

Flags are generated from dotted config paths and auto-typed from Pydantic field annotations:

• str / int / float → direct type coercion
• bool / Optional[bool] → store_true / store_false actions
• Literal["a", "b"] → restricted choices
• list[str] / set[int] → nargs="*"
• Optional[X] → nullable value

python main.py --key_00 from_cli --key_02.subkey_00 from_cli
# key_00 → "from_cli"  (CLI overrides all lower layers)
# key_02.subkey_00 → "from_cli"

Pass an explicit argument list via cli_args:

confstack.confstackify(Config, "myapp", parse_cli_args=True, cli_args=["--key_00", "test"])

Layer Merging

The diff utility shows exactly which values were overridden:

import confstack

config_default = AppCfg(
    key_00="[#!UNSET]",
    key_02=AppCfg.Key02(subkey_00="[#!UNSET]"),
    key_03=AppCfg.Key03(
        subkey_01=AppCfg.Key03.Subkey01(subsubkey_02="[#!UNSET]"),
    ),
)

config_confstackified = confstack.confstackify(AppCfg, "example_app", parse_cli_args=True)

print(confstack.diff(config_default, config_confstackified))

API Reference

confstackify(model_cls, app_name, overrides=None, config_file=None, parse_cli_args=False, cli_args=None)

Loads config from all layers and returns a validated model instance.

Parameter	Type	Description
model_cls	type[BaseModel]	Pydantic model class
app_name	str	App name (used for config file path and env var prefix)
overrides	dict | None	Nested dict or flat .-separated dict (layer 5)
config_file	str | None	Path to config file (default: ~/.config/{app_name}/config.json)
parse_cli_args	bool	If True, parse sys.argv (or cli_args) as CLI overrides
cli_args	list[str]|None	Explicit argument list for CLI parsing (overrides sys.argv)

ConfstackError

Raised when required fields are missing. Lists each field with its CLI flag and both env var names.

collect_config_paths(model_cls)

Returns all dotted config paths for a model, including nested sub-models.

>>> confstack.collect_config_paths(AppCfg)
['key_00', 'key_01', 'key_02.subkey_00', 'key_02.subkey_01', 'key_02.subkey_02',
 'key_02.subkey_03', 'key_03.subkey_00.subsubkey_00', 'key_03.subkey_01.subsubkey_00',
 'key_03.subkey_01.subsubkey_01', 'key_03.subkey_01.subsubkey_02']

unflatten(flat_dict, sep=".")

Converts a flat dotted dict to a nested dict.

>>> confstack.unflatten({"key_02.subkey_00": "val", "key_00": "x"})
{'key_02': {'subkey_00': 'val'}, 'key_00': 'x'}

diff(left, right)

Returns a coloured unified diff of two Pydantic model instances (requires Git).

before = confstack.confstackify(AppCfg, "myapp")
after = confstack.confstackify(AppCfg, "myapp", overrides={"key_00": "new"})
print(confstack.diff(before, after))

generate_config_mapping(model_cls, app_name)

Returns a pd.DataFrame mapping each config path to its default value, lowercase env name, and uppercase env name. Requires pandas (pip install pandas).

generate_config_markdown(model_cls, app_name, output_path=None)

Generates a formatted markdown table of all config paths. Requires htpy (pip install htpy). If output_path is None, writes to a .md file alongside the model's module.

Appendix: Dotted Environment Variables in Shell

# Set with `env`
env "my.var=value" ./script.sh

# Access
printenv "my.var"
python3 -c "import os; print(os.environ['my.var'])"