varlord 0.8.0

A powerful Python configuration management library with support for defaults, CLI args, environment variables, .env files, and optional etcd integration with dynamic updates

Varlord ⚙️

Stop wrestling with configuration chaos. Start with Varlord.

Varlord is a battle-tested Python configuration management library that eliminates the pain of managing configuration from multiple sources. Born from real-world production challenges, it provides a unified, type-safe, and elegant solution for configuration management.

🎯 The Problem We Solve

Real-World Configuration Nightmares

Every Python developer has faced these frustrating scenarios:

❌ The Configuration Spaghetti

# Your code becomes a mess of conditionals and parsing
host = os.getenv("HOST", "127.0.0.1")
port = int(os.getenv("PORT", "8000"))  # What if PORT is not a number?
debug = os.getenv("DEBUG", "false").lower() == "true"  # Really?
if "--host" in sys.argv:
    host = sys.argv[sys.argv.index("--host") + 1]  # Error-prone parsing
# ... and it gets worse with nested configs, validation, etc.

❌ Priority Confusion

"Does CLI override env? Or env overrides CLI? Wait, what about the config file? Which one wins?"

❌ Type Conversion Hell

# String "true" vs boolean True vs "1" vs 1
# "8000" vs 8000
# Missing values, None handling, type errors at runtime...

❌ The Restart Tax

"I just need to change one config value. Why do I have to restart the entire service?"

❌ Silent Failures

"The config looks wrong, but the app starts anyway. Users report bugs 3 hours later."

✅ Varlord's Solution

One unified interface. Multiple sources. Clear priority. Built-in diagnostics.

from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
from varlord import Config, sources

@dataclass(frozen=True)
class AppConfig:
    """Application configuration with clear structure and validation."""
    
    # Required field - must be provided
    api_key: str = field(metadata={"description": "API key for authentication"})
    
    # Optional fields with sensible defaults
    host: str = field(default="127.0.0.1", metadata={"description": "Server host address"})
    port: int = field(default=8000, metadata={"description": "Server port number"})
    debug: bool = field(default=False, metadata={"description": "Enable debug mode"})
    timeout: float = field(default=30.0, metadata={"description": "Request timeout in seconds"})
    hello_message: Optional[str] = field(
        default=None, metadata={"description": "Optional greeting message"}
    )

def main():
    # Define configuration sources with clear priority order
    # Priority: CLI (highest) > User Config > App Config > System Config > Env > Defaults (lowest)
    cfg = Config(
        model=AppConfig,
        sources=[
            # System-wide configuration (lowest priority, rarely overridden)
            sources.YAML("/etc/myapp/config.yaml"),  # System config
            
            # Application-level configuration
            sources.JSON(Path(__file__).parent / "config.json"),  # App directory
            
            # User-specific configuration (overrides system and app configs)
            sources.YAML(Path.home() / ".config" / "myapp" / "config.yaml"),  # User directory
            sources.TOML(Path.home() / ".myapp.toml"),  # Alternative user config
            
            # Environment variables (common in containers/CI)
            sources.Env(),
            sources.DotEnv(".env"),  # Local development
            
            # Command-line arguments (highest priority, for debugging/overrides)
            sources.CLI(),
        ],
    )
    
    # One line to add comprehensive CLI management: --help, --check-variables, etc.
    # This single call adds:
    #   - --help / -h: Auto-generated help from your model metadata
    #   - --check-variables / -cv: Complete configuration diagnostics
    #   - Automatic validation and error reporting
    #   - Exit handling (exits if help/cv is requested)
    cfg.handle_cli_commands()  # Handles --help, -cv automatically, exits if needed
    
    # Load configuration - type-safe, validated, ready to use
    app = cfg.load()
    
    # Your application code
    print(f"Starting server on {app.host}:{app.port}")
    print(f"Debug: {app.debug}, Timeout: {app.timeout}s")

if __name__ == "__main__":
    main()

What just happened?

1. ✅ Multiple Sources, Unified Interface: System config, app config, user config, env vars, CLI - all handled the same way
2. ✅ Clear Priority: Later sources override earlier ones - no confusion
3. ✅ Automatic Type Conversion: Strings from files/env → proper types (int, bool, float)
4. ✅ Model-Driven Filtering: Each source only reads fields defined in your model
5. ✅ Built-in Diagnostics: --check-variables shows exactly what's loaded from where
6. ✅ Zero Boilerplate: No parsing, no type conversion code, no priority logic

Try it:

# See comprehensive configuration diagnostics
python app.py --check-variables
# or short form
python app.py -cv

# See help with all sources and priority
python app.py --help

# Run normally
python app.py --api-key your_key

The --check-variables output shows everything:

When you run python app.py -cv, you get a comprehensive diagnostic report:

+---------------+----------+---------------+----------+-----------+
| Variable      | Required | Status        | Source   | Value     |
+---------------+----------+---------------+----------+-----------+
| api_key       | Required | Missing       | defaults | None      |
| host          | Optional | Loaded        | dotenv   | localhost |
| port          | Optional | Loaded        | dotenv   | 7000      |
| debug         | Optional | Loaded        | dotenv   | true      |
| timeout       | Optional | Loaded        | dotenv   | 20.0      |
| hello_message | Optional | Using Default | defaults | None      |
+---------------+----------+---------------+----------+-----------+

Configuration Source Priority and Details:

+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+
| Priority   | Source Name | Source ID | Instance                               | Status | Load Time (ms) | Watch Support | Last Update |
+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+
| 1 (lowest) | defaults    | defaults  | <Defaults(model=AppConfig)>            | Active | 0.00           | No            | N/A         |
| 2          | yaml        | yaml      | <YAML(/etc/myapp/config.yaml)>         | Active | 0.15           | No            | N/A         |
| 3          | json        | json      | <JSON(config.json)>                    | Active | 0.08           | No            | N/A         |
| 4          | yaml        | yaml      | <YAML(~/.config/myapp/config.yaml)>    | Active | 0.12           | No            | N/A         |
| 5          | toml        | toml      | <TOML(~/.myapp.toml)>                  | Active | 0.05           | No            | N/A         |
| 6          | env         | env       | <Env(model-based)>                     | Active | 0.05           | No            | N/A         |
| 7          | dotenv      | dotenv    | <DotEnv(.env)>                         | Active | 0.03           | No            | N/A         |
| 8 (highest)| cli         | cli       | <CLI()>                                | Active | 0.20           | No            | N/A         |
+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+


Note: Later sources override earlier ones (higher priority).

⚠️  Missing required fields: api_key
   Exiting with code 1. Please provide these fields and try again.
   For help, run: python app.py --help

What this tells you:

• Variable Status: See which fields are required vs optional, loaded vs missing
• Source Tracking: Know exactly which source (defaults/env/cli/file) provided each value
• Priority Order: Understand the resolution chain - later sources override earlier
• Performance: Load times for each source (useful for optimization)
• Validation: Missing required fields are caught immediately with clear error messages

Key Benefits:

• 🔍 Complete Visibility: See exactly which source provides each value - no more guessing where config comes from
• 📊 Priority Visualization: Understand the resolution order at a glance - see which source wins for each field
• ⚡ Performance Metrics: Load times for each source - identify slow config sources
• 🛡️ Validation: Missing required fields are caught before app starts - fail fast with clear errors
• 📝 Self-Documenting: Help text generated from your model metadata - no manual documentation needed
• 🎯 Zero Configuration: handle_cli_commands() adds all this with one line - no boilerplate

Real-World Scenarios:

• Debugging: "Why is my app using the wrong port?" → python app.py -cv shows port comes from env, not CLI
• Onboarding: New team member runs python app.py --help → sees all config options with descriptions
• CI/CD: Missing required field? → -cv shows exactly what's missing before deployment fails
• Multi-Environment: See which config file (system/user/app) is actually being used

That's it. No parsing, no type conversion, no priority confusion. Just clean, type-safe configuration with built-in diagnostics.

---

🌟 Why Varlord?

🎯 Core Value Propositions

Problem	Varlord Solution	Impact
Config scattered everywhere	Unified interface for all sources	Single source of truth
Priority confusion	Simple rule: later sources override earlier	Predictable behavior
Type conversion errors	Automatic conversion with validation	Catch errors early
No runtime updates	Optional etcd watch for dynamic updates	Zero-downtime config changes
Repetitive boilerplate	Model-driven, auto-filtering	90% less code
Silent failures	Built-in validation framework	Fail fast, fail clear

💡 Key Differentiators

1. 🎯 Model-Driven Design: Define your config once as a dataclass, and Varlord handles the rest
2. 🔄 Smart Auto-Filtering: Sources automatically filter by model fields - no prefix management needed
3. ⚡ Zero Boilerplate: Model defaults are automatic, model is auto-injected to sources
4. 🛡️ Type Safety First: Full type hints support with automatic conversion and validation
5. 🚀 Production Ready: Thread-safe, fail-safe, battle-tested in production environments

---

🚀 Quick Start

Installation

pip install varlord

# With optional features
pip install varlord[dotenv,etcd]

Basic Usage (30 seconds)

from dataclasses import dataclass, field
from varlord import Config, sources

@dataclass(frozen=True)
class AppConfig:
    host: str = field(default="127.0.0.1")
    port: int = field(default=8000)
    debug: bool = field(default=False)

# Create config - that's it!
cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),   # Reads HOST, PORT, DEBUG from environment
        sources.CLI(),   # Reads --host, --port, --debug from CLI
    ],
)

app = cfg.load()  # Type-safe, validated config object
print(f"Server: {app.host}:{app.port}, Debug: {app.debug}")

Run it:

# Use defaults
python app.py
# Output: Server: 127.0.0.1:8000, Debug: False

# Override with env
export HOST=0.0.0.0 PORT=9000
python app.py
# Output: Server: 0.0.0.0:9000, Debug: False

# Override with CLI (highest priority)
python app.py --host 192.168.1.1 --port 8080 --debug
# Output: Server: 192.168.1.1:8080, Debug: True

One-Liner Convenience Method

# Even simpler for common cases
cfg = Config.from_model(AppConfig, cli=True, dotenv=".env")
app = cfg.load()

---

💼 Real-World Use Cases

Use Case 1: Microservice Configuration

Problem: Your microservice needs config from multiple sources, and you're tired of writing parsing code.

Solution:

@dataclass(frozen=True)
class ServiceConfig:
    db_host: str = field(default="localhost")
    db_port: int = field(default=5432)
    api_key: str = field()  # Required - must be provided
    log_level: str = field(default="INFO")
    max_workers: int = field(default=4)

cfg = Config(
    model=ServiceConfig,
    sources=[
        sources.Env(),           # Production: from environment
        sources.DotEnv(".env"),  # Development: from .env file
        sources.CLI(),           # Override: from command line
    ],
)

config = cfg.load()  # Validated, type-safe, ready to use

Benefits:

• ✅ Same code works in dev (.env), staging (env vars), and prod (env vars)
• ✅ CLI overrides for debugging: python service.py --log-level DEBUG
• ✅ Type safety: max_workers is always an int, never a string
• ✅ Validation: Missing api_key fails fast with clear error

Use Case 2: Dynamic Configuration Updates

Problem: You need to change configuration without restarting the service.

Solution:

def on_config_change(new_config, diff):
    print(f"Config updated: {diff}")
    # Update your app's behavior based on new config

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),
        sources.Etcd(
            host="etcd.example.com",
            prefix="/app/config/",
            watch=True,  # Enable dynamic updates
        ),
    ],
)

store = cfg.load_store()  # Returns ConfigStore for dynamic updates
store.subscribe(on_config_change)

# Thread-safe access to current config
current = store.get()

Benefits:

• ✅ Zero-downtime configuration updates
• ✅ Thread-safe concurrent access
• ✅ Automatic validation on updates
• ✅ Change notifications via callbacks

Use Case 3: Multi-Environment Deployment

Problem: Different configs for dev, staging, and production, but you want one codebase.

Solution:

# Development: .env file
# Staging: Environment variables
# Production: etcd + environment variables

cfg = Config(
    model=AppConfig,
    sources=[
        sources.DotEnv(".env"),  # Dev only (file may not exist in prod)
        sources.Env(),            # All environments
        sources.Etcd.from_env() if os.getenv("ETCD_HOST") else None,  # Prod only
        sources.CLI(),           # Override for debugging
    ],
)

Benefits:

• ✅ One codebase, multiple environments
• ✅ Environment-specific sources automatically handled
• ✅ Clear priority: CLI > etcd > env > .env > defaults

Use Case 4: Complex Nested Configuration

Problem: Your config has nested structures (database, cache, API keys, etc.).

Solution:

@dataclass(frozen=True)
class DatabaseConfig:
    host: str = field(default="localhost")
    port: int = field(default=5432)
    name: str = field(default="mydb")

@dataclass(frozen=True)
class AppConfig:
    db: DatabaseConfig = field(default_factory=DatabaseConfig)
    api_key: str = field()
    cache_ttl: int = field(default=3600)

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Reads DB__HOST, DB__PORT, DB__NAME automatically
        sources.CLI(),  # Reads --db-host, --db-port, etc.
    ],
)

config = cfg.load()
# Access: config.db.host, config.db.port, config.api_key

Benefits:

• ✅ Automatic nested key mapping (DB__HOST → db.host)
• ✅ Type-safe nested access
• ✅ Validation at all levels

---

🎨 Key Features

1. Multiple Sources, Unified Interface

sources = [
    sources.Defaults(),      # From model defaults (automatic)
    sources.Env(),           # From environment variables
    sources.CLI(),           # From command-line arguments
    sources.DotEnv(".env"),  # From .env files
    sources.YAML("config.yaml"),  # From YAML files
    sources.TOML("config.toml"),  # From TOML files
    sources.Etcd(...),       # From etcd (optional)
]

2. Simple Priority Rule

Later sources override earlier ones. That's it.

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),   # Priority 1 (lowest)
        sources.CLI(),   # Priority 2 (highest - overrides env)
    ],
)

3. Automatic Type Conversion

# Environment variables are strings, but Varlord converts them automatically
export PORT=9000 DEBUG=true TIMEOUT=30.5

@dataclass(frozen=True)
class Config:
    port: int = 8000        # "9000" → 9000
    debug: bool = False     # "true" → True
    timeout: float = 30.0   # "30.5" → 30.5

4. Model-Driven Filtering

# Your model defines what config you need
@dataclass(frozen=True)
class Config:
    host: str = "127.0.0.1"
    port: int = 8000
    # ... only these fields

# Sources automatically filter - no prefix management needed
# Env source only reads HOST and PORT, ignores everything else
# CLI source only parses --host and --port, ignores other args

5. Built-in Validation

from varlord.validators import validate_range, validate_regex

@dataclass(frozen=True)
class Config:
    port: int = field(default=8000)
    host: str = field(default="127.0.0.1")
    
    def __post_init__(self):
        validate_range(self.port, min=1, max=65535)
        validate_regex(self.host, r'^\d+\.\d+\.\d+\.\d+$')

6. Dynamic Updates (Optional)

store = cfg.load_store()  # Enable watch if sources support it
store.subscribe(lambda new_config, diff: print(f"Updated: {diff}"))

# Config updates automatically in background
# Thread-safe access: current = store.get()

---

📚 Documentation

• 📖 Full Documentation: https://varlord.readthedocs.io
• 🚀 Quick Start Guide: Quick Start
• 💡 Examples: Examples Directory
• 🎯 API Reference: API Documentation

---

🧠 Memory Aids (Quick Reference)

The Varlord Mantra

"Define once, use everywhere. Later overrides earlier. Types are automatic."

Priority Cheat Sheet

Defaults < .env < Environment < YAML/TOML < etcd < CLI
  (lowest priority)                          (highest priority)

Common Patterns

# Pattern 1: Simple (most common)
Config(model=AppConfig, sources=[sources.Env(), sources.CLI()])

# Pattern 2: With .env file
Config(model=AppConfig, sources=[sources.DotEnv(".env"), sources.Env(), sources.CLI()])

# Pattern 3: Dynamic updates
Config(model=AppConfig, sources=[sources.Env(), sources.Etcd(..., watch=True)])
store = cfg.load_store()

# Pattern 4: One-liner
Config.from_model(AppConfig, cli=True, dotenv=".env")

---

🏢 Production Proven

Varlord is part of the Agentsmith ecosystem, battle-tested in production environments:

• ✅ Deployed in multiple highway management companies
• ✅ Used by securities firms and regulatory agencies
• ✅ Handles high-throughput microservices
• ✅ Thread-safe and production-ready

🌟 Agentsmith Open-Source Projects

• Varlord ⚙️ - Configuration management (this project)
• Routilux ⚡ - Event-driven workflow orchestration
• Serilux 📦 - Flexible serialization framework
• Lexilux 🚀 - Unified LLM API client

---

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

📄 License

Licensed under the Apache License 2.0. See LICENSE for details.

---

🎯 TL;DR

Varlord solves configuration management once and for all:

1. ✅ Define your config as a dataclass - type-safe, validated
2. ✅ Add sources in priority order - later overrides earlier
3. ✅ Call load() - get a type-safe config object
4. ✅ Optional: Enable dynamic updates - zero-downtime config changes

No more parsing. No more type conversion. No more priority confusion.

# Before: 50+ lines of parsing, type conversion, validation
# After: 3 lines
cfg = Config(model=AppConfig, sources=[sources.Env(), sources.CLI()])
app = cfg.load()

That's the Varlord promise. 🚀