Skip to main content

Pydantic settings from HashiCorp Vault KV and JSON with caching and retry

Project description

vltconfig

Pydantic settings from HashiCorp Vault KV and JSON with caching and retry

PyPI version Python 3.11+ License: MIT

Features

  • Multi-source configuration: Load from Vault KV v2, JSON files, and environment variables
  • 🔒 Type-safe: Full pydantic v2 validation with complete type hints
  • 🔁 Retry logic: Automatic retries with exponential backoff for transient failures
  • 💾 Caching: In-memory cache with configurable TTL (5 min default)
  • 🔐 Multiple auth methods: Token, AppRole, and Username/Password authentication
  • 🎯 Fallback support: Automatic fallback between authentication methods
  • 📝 Comprehensive logging: Structured logging with loguru
  • Well tested: 96%+ test coverage with 53 tests

Installation

pip install vltconfig

Quick Start

from vltconfig import VaultJsonConfig
from pydantic import Field

# Define your configuration model
class AppConfig(VaultJsonConfig):
    database_url: str = Field(description="Database connection string")
    api_key: str = Field(description="API key for external service")
    debug: bool = Field(default=False, description="Debug mode")
    max_connections: int = Field(default=10, description="Max DB connections")

# Configuration is automatically loaded from:
# 1. Environment variables (highest priority)
# 2. HashiCorp Vault KV store
# 3. JSON file (config.json)
# 4. Default values (lowest priority)
config = AppConfig()

print(config.database_url)
print(config.api_key)

Authentication

Token Authentication

import os

os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_TOKEN"] = "s.your-vault-token"
os.environ["VAULT_APP_NAME"] = "myapp/config"

config = AppConfig()

AppRole Authentication

import os

os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_ROLE_ID"] = "your-role-id"
os.environ["VAULT_SECRET_ID"] = "your-secret-id"
os.environ["VAULT_APP_NAME"] = "myapp/config"

config = AppConfig()

Username/Password Authentication

import os

os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_USERNAME"] = "readonly"
os.environ["VAULT_PASSWORD"] = "your-password"
os.environ["VAULT_APP_NAME"] = "myapp/config"

config = AppConfig()

Authentication Fallback

If multiple authentication methods are configured, vltconfig will automatically try them in order:

  1. Token authentication (VAULT_TOKEN)
  2. AppRole authentication (VAULT_ROLE_ID + VAULT_SECRET_ID)
  3. Username/Password authentication (VAULT_USERNAME + VAULT_PASSWORD)

This provides resilience if one authentication method is temporarily unavailable.

Configuration

Environment Variables

Required

  • VAULT_ADDRESS: URL of the Vault server (e.g., https://vault.example.com)
  • VAULT_APP_NAME: Path to secrets in Vault KV store (e.g., myapp/config)

Authentication (provide at least one method)

  • VAULT_TOKEN: Vault authentication token
  • VAULT_USERNAME + VAULT_PASSWORD: Username/password authentication
  • VAULT_ROLE_ID + VAULT_SECRET_ID: AppRole authentication

Optional

  • VAULT_MOUNT_POINT: Mount point for Vault KV engine (default: secret)
  • PYDANTIC_JSON_PATH: Path to directory containing config.json
  • VAULT_CACHE_DISABLED: Set to true to disable caching
  • VAULT_CACHE_TTL: Cache TTL in seconds (default: 300)

Caching

vltconfig caches secrets from Vault for 5 minutes by default to reduce load on Vault servers:

import os

# Disable caching
os.environ["VAULT_CACHE_DISABLED"] = "true"

# Or set custom TTL (in seconds)
os.environ["VAULT_CACHE_TTL"] = "600"  # 10 minutes

config = AppConfig()

Retry Logic

Transient failures are automatically retried with exponential backoff:

  • Retries: Up to 3 attempts
  • Backoff: Exponential (2s, 4s, 8s)
  • Retryable errors: Vault server down, connection errors, timeouts
  • Non-retryable errors: Authentication failures, permission errors, invalid paths

Custom Mount Point

If your Vault KV engine uses a custom mount point:

import os

os.environ["VAULT_MOUNT_POINT"] = "custom-kv"
os.environ["VAULT_APP_NAME"] = "myapp/config"

config = AppConfig()

JSON Configuration

You can also load configuration from a JSON file:

import os

# Create config.json
os.makedirs("config", exist_ok=True)
with open("config/config.json", "w") as f:
    f.write('{"database_url": "postgresql://localhost/mydb", "debug": true}')

# Point to config directory
os.environ["PYDANTIC_JSON_PATH"] = "./config"

config = AppConfig()

Configuration Priority

Values are loaded in the following priority order (highest to lowest):

  1. Environment variables (highest priority)
  2. Vault KV store
  3. JSON file (config.json)
  4. Initialization parameters
  5. .env file
  6. Default values (lowest priority)

How It Works

vltconfig extends pydantic-settings to add custom settings sources:

from vltconfig import VaultJsonConfig

class AppConfig(VaultJsonConfig):
    # Your fields here
    pass

# When you instantiate AppConfig(), it:
# 1. Connects to Vault using configured authentication
# 2. Fetches secrets from the specified path
# 3. Loads config.json if available
# 4. Merges all sources according to priority
# 5. Validates using pydantic
# 6. Returns type-safe configuration object

Migration from vault-pydantic-simple

⚠️ Breaking Change in v1.0.0: The import path has changed.

If you're upgrading from vault-pydantic-simple:

# Uninstall old package
pip uninstall vault-pydantic-simple

# Install new package
pip install vltconfig

Update your imports:

# Old (v0.x)
from vlt import VaultJsonConfig

# New (v1.0.0+)
from vltconfig import VaultJsonConfig

Requirements

  • Python >= 3.11
  • pydantic >= 2.10.6
  • pydantic-settings >= 2.7.1
  • hvac >= 2.3.0 (HashiCorp Vault client)
  • loguru >= 0.7.3 (logging)
  • tenacity >= 9.0.0 (retry logic)

Examples

The repository includes practical examples in the examples/ directory:

📝 Available Examples

  1. debug_example.py - Complete debugging setup

    • Shows how to enable detailed TRACE logging
    • Demonstrates custom JSON config path setup
    • Full example with all configuration sources
  2. json_only_example.py - JSON-only configuration

    • Using vltconfig without Vault (local development)
    • Shows graceful fallback when Vault is not configured
    • Environment variable override examples

🚀 Running Examples

# Clone the repository
git clone https://github.com/tofuurem/vltconfig.git
cd vltconfig

# Run the debug example
python examples/debug_example.py

# Run the JSON-only example
python examples/json_only_example.py

📂 Example Config Files

The examples/config/ directory contains sample config.json files you can use as templates:

{
  "database_url": "postgresql://localhost:5432/mydb",
  "api_key": "your-api-key-here",
  "debug": true,
  "max_connections": 20
}

Development

# Clone repository
git clone https://github.com/tofuurem/vltconfig.git
cd vltconfig

# Install dependencies with uv
uv sync --group dev

# Run tests
uv run pytest --cov=vltconfig

# Run linters
uv run ruff check .
uv run mypy vltconfig/

Testing

# Run all tests
uv run pytest

# Run with coverage
uv run pytest --cov=vltconfig --cov-report=term-missing

# Run specific test file
uv run pytest tests/unit/test_config.py -v

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details.

Links

Author

tofuurem - rabbit_1399@icloud.com


Made with ❤️ using Python and Pydantic

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vltconfig-1.0.1.tar.gz (12.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

vltconfig-1.0.1-py3-none-any.whl (10.9 kB view details)

Uploaded Python 3

File details

Details for the file vltconfig-1.0.1.tar.gz.

File metadata

  • Download URL: vltconfig-1.0.1.tar.gz
  • Upload date:
  • Size: 12.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for vltconfig-1.0.1.tar.gz
Algorithm Hash digest
SHA256 df6f1017692cbcf542b96b765127526b1a5df3355c85d5bcd8136ed29188821d
MD5 a7a4541f700367675e89602775a84a22
BLAKE2b-256 a311b7b2a5e813d69b9e286340ab2a19315ad6fa47bcb7979991a8d8bbacb842

See more details on using hashes here.

Provenance

The following attestation bundles were made for vltconfig-1.0.1.tar.gz:

Publisher: main.yml on tofuurem/vltconfig

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vltconfig-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: vltconfig-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 10.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for vltconfig-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 e2d4d84875f8459541f98ea5874705c78318908d4ffdb97804aaac07492566f1
MD5 796e27f341dddc5d7d9e1308cb2c52a9
BLAKE2b-256 b865a899ecac1a219a1529d9d6193d61c194604365b73f746c85edb7f00dff17

See more details on using hashes here.

Provenance

The following attestation bundles were made for vltconfig-1.0.1-py3-none-any.whl:

Publisher: main.yml on tofuurem/vltconfig

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page