configmerge 1.0.0

Load, merge, and validate configuration from JSON, YAML, TOML, and environment variables.

configmerge

Load, merge, and validate configuration from JSON, YAML, TOML, and environment variables.

The Problem

Configuration management in Python is fragmented:

# You need different libraries for different formats 😤
import json
import yaml  # pip install pyyaml
import tomllib  # Python 3.11+ only
import os

# Manual loading and merging 😫
with open("base.json") as f:
    base_config = json.load(f)

with open("local.yaml") as f:
    local_config = yaml.safe_load(f)

# Manual environment variable handling 😵
config = {**base_config, **local_config}
if os.getenv("DATABASE_URL"):
    config["database"]["url"] = os.getenv("DATABASE_URL")

The Solution

from configmerge import load

# Load and merge multiple formats + environment variables! 🎉
config = load(
    "config.json",           # Base configuration
    "config.local.yaml",     # Local overrides  
    "config.prod.toml",      # Production settings
    env_prefix="APP_"        # Environment variables (APP_*)
)

# Access nested configuration easily
db_host = config["database"]["host"]

Installation

# Core functionality (JSON + .env files)
pip install configmerge

# With YAML support
pip install configmerge[yaml]

# With TOML support  
pip install configmerge[toml]

# With all format support
pip install configmerge[all]

Features

• Multi-format support: JSON, YAML, TOML, .env files
• Zero core dependencies: JSON and .env work out of the box
• Deep merging: Nested dictionaries merged intelligently
• Environment integration: Load env vars with nested key support
• Type preservation: Automatic type conversion from env vars
• Error handling: Graceful handling of missing files
• Python 3.10+: Modern Python with full type hints

Quick Start

Basic Usage

from configmerge import load, load_file, merge

# Load single file
config = load_file("config.json")

# Load and merge multiple files
config = load("base.yaml", "local.yaml", "prod.yaml")

# Include environment variables
config = load("config.yaml", env_prefix="APP_")

Supported Formats

from configmerge import load_file

# JSON (always supported)
config = load_file("config.json")

# YAML (requires pyyaml)
config = load_file("config.yaml")

# TOML (requires tomli on Python < 3.11)
config = load_file("config.toml")

# .env files (always supported)
config = load_file(".env")

Environment Variables

from configmerge import from_env

# Set environment variables
# APP_DATABASE__HOST=localhost
# APP_DATABASE__PORT=5432
# APP_DEBUG=true

config = from_env("APP_")
# Result: {
#   "database": {"host": "localhost", "port": 5432},
#   "debug": True
# }

Advanced Usage

Deep Merging

from configmerge import merge

base = {
    "database": {"host": "localhost", "port": 5432},
    "cache": {"ttl": 300}
}

override = {
    "database": {"port": 3306},  # Override port, keep host
    "debug": True                # Add new key
}

result = merge(base, override)
# Result: {
#   "database": {"host": "localhost", "port": 3306},
#   "cache": {"ttl": 300},
#   "debug": True
# }

Configuration Layers

from configmerge import load

# Load configuration in priority order (later overrides earlier)
config = load(
    "config/default.yaml",      # 1. Default settings
    "config/production.yaml",   # 2. Environment-specific
    "config/local.yaml",        # 3. Local overrides
    env_prefix="MYAPP_"         # 4. Environment variables (highest priority)
)

Environment Variable Mapping

Environment variables use __ (double underscore) for nested keys:

# Environment variables
export MYAPP_DATABASE__HOST=prod-db.example.com
export MYAPP_DATABASE__PORT=5432
export MYAPP_CACHE__REDIS__URL=redis://localhost:6379
export MYAPP_DEBUG=false
export MYAPP_WORKERS=4

from configmerge import from_env

config = from_env("MYAPP_")
# Result: {
#   "database": {
#     "host": "prod-db.example.com",
#     "port": 5432
#   },
#   "cache": {
#     "redis": {
#       "url": "redis://localhost:6379"
#     }
#   },
#   "debug": False,
#   "workers": 4
# }

Type Conversion

Environment variables are automatically converted to appropriate types:

export APP_DEBUG=true          # → Boolean: True
export APP_PORT=8080          # → Integer: 8080  
export APP_RATE=1.5           # → Float: 1.5
export APP_TAGS='["a","b"]'   # → List: ["a", "b"]
export APP_CONFIG='{"x":1}'   # → Dict: {"x": 1}
export APP_NAME=myapp         # → String: "myapp"

Configuration Examples

Web Application

# config/default.yaml
database:
  host: localhost
  port: 5432
  name: myapp
  
server:
  host: 0.0.0.0
  port: 8000
  
cache:
  ttl: 300

# config/production.yaml  
database:
  host: prod-db.example.com
  
server:
  port: 80

# Load configuration
from configmerge import load

config = load(
    "config/default.yaml",
    "config/production.yaml", 
    env_prefix="APP_"
)

# Use configuration
app.run(
    host=config["server"]["host"],
    port=config["server"]["port"]
)

Docker Configuration

# Dockerfile
ENV APP_DATABASE__HOST=postgres
ENV APP_DATABASE__PORT=5432
ENV APP_REDIS__URL=redis://redis:6379

# app.py
from configmerge import load

config = load(
    "config.yaml",        # Base configuration
    env_prefix="APP_"     # Docker environment variables
)

12-Factor App

from configmerge import load

# Follow 12-factor app principles
config = load(
    "config/default.json",    # Defaults in code
    env_prefix="APP_"         # Config from environment
)

# All configuration comes from environment in production
DATABASE_URL = config["database"]["url"]
SECRET_KEY = config["security"]["secret_key"]

API Reference

load(*paths, env_prefix=None)

Load and merge configuration from multiple sources.

Parameters:

• *paths: Configuration file paths
• env_prefix: Environment variable prefix (optional)

Returns: Merged configuration dictionary

load_file(path)

Load configuration from a single file.

Parameters:

• path: Path to configuration file

Returns: Configuration dictionary

merge(base, override)

Deep merge two configuration dictionaries.

Parameters:

• base: Base configuration
• override: Override configuration

Returns: Merged configuration dictionary

from_env(prefix)

Load configuration from environment variables.

Parameters:

• prefix: Environment variable prefix

Returns: Configuration dictionary

Error Handling

from configmerge import load, load_file

# Missing files are silently ignored
config = load("missing.yaml", "existing.yaml")  # Only loads existing.yaml

# Unknown formats raise ValueError
try:
    config = load_file("config.xml")
except ValueError as e:
    print(f"Unsupported format: {e}")

# Missing dependencies raise ImportError
try:
    config = load_file("config.yaml")  # Without pyyaml installed
except ImportError as e:
    print(f"Install pyyaml: {e}")

Best Practices

1. Layer Configuration

# Good: Clear priority order
config = load(
    "config/defaults.yaml",     # Lowest priority
    "config/environment.yaml",  # Environment-specific
    "config/local.yaml",        # Local development
    env_prefix="APP_"           # Highest priority
)

2. Use Environment Variables for Secrets

# config.yaml - No secrets in files!
database:
  host: localhost
  port: 5432
  # Don't put passwords in config files!

# Environment variables for secrets
# APP_DATABASE__PASSWORD=secret123
config = load("config.yaml", env_prefix="APP_")

3. Validate Configuration

from configmerge import load

config = load("config.yaml", env_prefix="APP_")

# Validate required settings
required_keys = ["database.host", "database.port", "secret_key"]
for key in required_keys:
    keys = key.split(".")
    value = config
    for k in keys:
        value = value.get(k)
        if value is None:
            raise ValueError(f"Missing required config: {key}")

Comparison

Feature	configmerge	python-decouple	dynaconf	python-dotenv
Multi-format	✅ JSON/YAML/TOML/.env	❌ .env only	✅ Many formats	❌ .env only
Deep merging	✅	❌	✅	❌
Zero dependencies	✅ Core features	✅	❌	✅
Type conversion	✅ Automatic	✅ Manual	✅ Automatic	❌
Nested env vars	✅ __ separator	❌	✅	❌

Requirements

• Python 3.10+
• Optional: pyyaml (for YAML support)
• Optional: tomli (for TOML support on Python < 3.11)

License

MIT License - Free for commercial use

Contributing

Contributions welcome! Please see our Contributing Guide.

Related Projects

• asyncbridge - Async/sync conversion utilities
• httpx-defaults - Production HTTP client
• devkitx - Security-first Python utilities