varlord 0.4.0

A powerful Python configuration management library with support for defaults, CLI args, environment variables, .env files, and optional etcd integration with dynamic updates

Varlord ⚙️

Varlord is a powerful Python configuration management library that provides a unified interface for loading configuration from multiple sources with customizable priority ordering and optional dynamic updates via etcd.

✨ Features

• 🔧 Multiple Sources: Support for defaults (automatic), CLI arguments, environment variables, .env files, and optional etcd integration
• 🎯 Simple Priority: Priority determined by sources order (later overrides earlier)
• 🔄 Dynamic Updates: Real-time configuration updates via etcd watch (optional)
• 🛡️ Type Safety: Built-in support for dataclass models with automatic type conversion
• 📝 Logging Support: Configurable logging to track configuration loading and merging
• ✅ Validation Framework: Built-in value validators and required field validation with comprehensive error messages
• 🔍 Model-Driven Filtering: All sources automatically filter by model fields - no prefix needed
• 🔌 Pluggable Architecture: Clean source abstraction for easy extension
• 📦 Optional Dependencies: Lightweight core with optional extras for dotenv and etcd
• 🚀 Production Ready: Thread-safe, fail-safe update strategies, and comprehensive error handling
• 🎨 Simple API: Convenience methods and auto-injection for cleaner code

📦 Installation

Basic Installation

pip install varlord

With Optional Dependencies

# With .env file support
pip install varlord[dotenv]

# With etcd support
pip install varlord[etcd]

# With all optional dependencies
pip install varlord[dotenv,etcd]

# Development installation
pip install -e ".[dev]"

Development Setup with uv (Recommended)

This project uses uv for fast dependency management. Install uv first:

curl -LsSf https://astral.sh/uv/install.sh | sh

Then set up the development environment:

Recommended: For active development

# Install package with all development dependencies (recommended)
make dev-install

# Or manually with uv (dev group is installed by default)
uv sync --group docs --all-extras

Alternative: Dependencies only (for CI/CD or code review)

# Create virtual environment and install dependencies only (without installing the package)
# Useful for: CI/CD pipelines, code review, or when you only need development tools
make setup-venv

# Later, if you need to install the package:
make install

Understanding dependency groups vs extras:

• Dependency groups (dev, docs): Development dependencies that are not published to PyPI. The dev group is installed by default with uv sync.
• Extras (dotenv, etcd): Optional runtime features that are published to PyPI.

All make commands will automatically use uv if available, otherwise fall back to pip.

🚀 Quick Start

Basic Usage

from dataclasses import dataclass, field
from varlord import Config, sources

@dataclass(frozen=True)
class AppConfig:
    host: str = field(default="127.0.0.1")  # Optional (has default)
    port: int = field(default=8000)  # Optional (has default)
    debug: bool = field(default=False)  # Optional (has default)

# Model defaults are automatically applied - no need for sources.Defaults
# Model is automatically injected to all sources - no need to pass model parameter
cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),        # HOST, PORT, DEBUG (model auto-injected)
        sources.CLI(),        # --host, --port, --debug (model auto-injected)
    ],
)

app = cfg.load()
print(app.host)  # Can be overridden by env var or CLI arg
print(app.port)

Convenience Method

# One-line setup for common cases
cfg = Config.from_model(
    AppConfig,
    cli=True,
    dotenv=".env",
)

app = cfg.load()

Priority Ordering

Method 1: Reorder sources (recommended - simplest)

# Priority is determined by sources order: later sources override earlier ones
# Model defaults are automatically applied first (lowest priority)
cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Overrides defaults
        sources.CLI(),  # Highest priority (overrides env)
    ],
)

Method 2: Use PriorityPolicy (advanced: per-key rules)

from varlord import PriorityPolicy

# Use when you need different priority rules for different keys
cfg = Config(
    model=AppConfig,
    sources=[...],
    policy=PriorityPolicy(
        default=["defaults", "env", "cli"],  # Default order for all keys
        overrides={
            "secrets.*": ["defaults", "etcd"],  # Secrets: skip env, only etcd can override
        }
    ),
)

Dynamic Updates with Etcd

def on_change(new_config, diff):
    print("Config updated:", diff)

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Defaults applied automatically
        sources.Etcd("http://127.0.0.1:2379", prefix="/app/", watch=True),
    ],
)

store = cfg.load_store()  # Automatically enables watch if sources support it
store.subscribe(on_change)

current = store.get()  # Thread-safe access to current config

Logging

Enable debug logging to track configuration loading:

import logging
from varlord import set_log_level

set_log_level(logging.DEBUG)
cfg = Config(...)
app = cfg.load()  # Logs source loads, merges, type conversions

Validation

Value Validation: Add validators in your model's __post_init__:

from dataclasses import dataclass, field
from varlord.validators import validate_range, validate_regex

@dataclass(frozen=True)
class AppConfig:
    port: int = field(default=8000)  # Optional (has default)
    host: str = field(default="127.0.0.1")  # Optional (has default)

    def __post_init__(self):
        validate_range(self.port, min=1, max=65535)
        validate_regex(self.host, r'^\d+\.\d+\.\d+\.\d+$')

Required Field Validation: Fields are automatically determined as required/optional:

• Fields without defaults and not Optional[T] are required
• Fields with Optional[T] type annotation are optional
• Fields with defaults (or default_factory) are optional

from dataclasses import dataclass, field
from typing import Optional
from varlord.model_validation import RequiredFieldError

@dataclass(frozen=True)
class AppConfig:
    api_key: str = field()  # Required (no default, not Optional)
    timeout: Optional[int] = field()  # Optional (Optional type)
    host: str = field(default="127.0.0.1")  # Optional (has default)

cfg = Config(model=AppConfig, sources=[])
try:
    app = cfg.load()  # Raises RequiredFieldError if api_key not provided
except RequiredFieldError as e:
    print(f"Missing required fields: {e.missing_fields}")

📚 Documentation

Full documentation is available at https://varlord.readthedocs.io

• Latest: https://varlord.readthedocs.io/en/latest/
• Stable: https://varlord.readthedocs.io/en/stable/

🎯 Key Concepts

Configuration Model

Use dataclass to define your configuration structure with type hints and default values.

Sources

Each source implements a unified interface:

• load() -> Mapping[str, Any]: Load configuration snapshot
• watch() -> Iterator[ChangeEvent] (optional): Stream of changes for dynamic updates
• name: Source name for identification

Priority Ordering

Simple (Recommended): Reorder sources list - later sources override earlier ones.

Advanced: Use PriorityPolicy for per-key priority rules (e.g., different rules for secrets).

Type Conversion

Automatic conversion from strings (env vars, CLI) to model field types (int, float, bool, etc.).

Validation

Value Validation: Add validators in your model's __post_init__ method to validate configuration values.

Required Field Validation: Fields are automatically determined as required/optional based on type annotation and default values. Required fields are automatically validated before __post_init__ is called.

ConfigStore

Runtime configuration management with:

• Thread-safe atomic snapshots
• Dynamic updates via watch mechanism
• Change subscriptions

🏢 About Agentsmith

Varlord is part of the Agentsmith open-source ecosystem. Agentsmith is a ToB AI agent and algorithm development platform, currently deployed in multiple highway management companies, securities firms, and regulatory agencies in China. The Agentsmith team is gradually open-sourcing the platform by removing proprietary code and algorithm modules, as well as enterprise-specific customizations, while decoupling the system for modular use by the open-source community.

🌟 Agentsmith Open-Source Projects

• Varlord ⚙️ - Configuration management library with multi-source support
• Routilux ⚡ - Event-driven workflow orchestration framework
• Serilux 📦 - Flexible serialization framework for Python objects
• Lexilux 🚀 - Unified LLM API client library

These projects are modular components extracted from the Agentsmith platform, designed to be used independently or together to build powerful applications.

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

📄 License

Licensed under the Apache License 2.0. See LICENSE for details.