Pydantic settings from HashiCorp Vault KV and JSON with caching and retry
Project description
vltconfig
Pydantic settings from HashiCorp Vault KV and JSON with caching and retry
Features
- ✨ Multi-source configuration: Load from Vault KV v2, JSON files, and environment variables
- 🔒 Type-safe: Full pydantic v2 validation with complete type hints
- 🔁 Retry logic: Automatic retries with exponential backoff for transient failures
- 💾 Caching: In-memory cache with configurable TTL (5 min default)
- 🔐 Multiple auth methods: Token, AppRole, and Username/Password authentication
- 🎯 Fallback support: Automatic fallback between authentication methods
- 📝 Comprehensive logging: Structured logging with loguru
- ✅ Well tested: 96%+ test coverage with 53 tests
Installation
pip install vltconfig
Quick Start
from vltconfig import VaultJsonConfig
from pydantic import Field
# Define your configuration model
class AppConfig(VaultJsonConfig):
database_url: str = Field(description="Database connection string")
api_key: str = Field(description="API key for external service")
debug: bool = Field(default=False, description="Debug mode")
max_connections: int = Field(default=10, description="Max DB connections")
# Configuration is automatically loaded from:
# 1. Environment variables (highest priority)
# 2. HashiCorp Vault KV store
# 3. JSON file (config.json)
# 4. Default values (lowest priority)
config = AppConfig()
print(config.database_url)
print(config.api_key)
Authentication
Token Authentication
import os
os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_TOKEN"] = "s.your-vault-token"
os.environ["VAULT_APP_NAME"] = "myapp/config"
config = AppConfig()
AppRole Authentication
import os
os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_ROLE_ID"] = "your-role-id"
os.environ["VAULT_SECRET_ID"] = "your-secret-id"
os.environ["VAULT_APP_NAME"] = "myapp/config"
config = AppConfig()
Username/Password Authentication
import os
os.environ["VAULT_ADDRESS"] = "https://vault.example.com"
os.environ["VAULT_USERNAME"] = "readonly"
os.environ["VAULT_PASSWORD"] = "your-password"
os.environ["VAULT_APP_NAME"] = "myapp/config"
config = AppConfig()
Authentication Fallback
If multiple authentication methods are configured, vltconfig will automatically try them in order:
- Token authentication (
VAULT_TOKEN) - AppRole authentication (
VAULT_ROLE_ID+VAULT_SECRET_ID) - Username/Password authentication (
VAULT_USERNAME+VAULT_PASSWORD)
This provides resilience if one authentication method is temporarily unavailable.
Configuration
Environment Variables
Required
VAULT_ADDRESS: URL of the Vault server (e.g.,https://vault.example.com)VAULT_APP_NAME: Path to secrets in Vault KV store (e.g.,myapp/config)
Authentication (provide at least one method)
VAULT_TOKEN: Vault authentication tokenVAULT_USERNAME+VAULT_PASSWORD: Username/password authenticationVAULT_ROLE_ID+VAULT_SECRET_ID: AppRole authentication
Optional
VAULT_MOUNT_POINT: Mount point for Vault KV engine (default:secret)PYDANTIC_JSON_PATH: Path to directory containingconfig.jsonVAULT_CACHE_DISABLED: Set totrueto disable cachingVAULT_CACHE_TTL: Cache TTL in seconds (default:300)
Caching
vltconfig caches secrets from Vault for 5 minutes by default to reduce load on Vault servers:
import os
# Disable caching
os.environ["VAULT_CACHE_DISABLED"] = "true"
# Or set custom TTL (in seconds)
os.environ["VAULT_CACHE_TTL"] = "600" # 10 minutes
config = AppConfig()
Retry Logic
Transient failures are automatically retried with exponential backoff:
- Retries: Up to 3 attempts
- Backoff: Exponential (2s, 4s, 8s)
- Retryable errors: Vault server down, connection errors, timeouts
- Non-retryable errors: Authentication failures, permission errors, invalid paths
Custom Mount Point
If your Vault KV engine uses a custom mount point:
import os
os.environ["VAULT_MOUNT_POINT"] = "custom-kv"
os.environ["VAULT_APP_NAME"] = "myapp/config"
config = AppConfig()
JSON Configuration
You can also load configuration from a JSON file:
import os
# Create config.json
os.makedirs("config", exist_ok=True)
with open("config/config.json", "w") as f:
f.write('{"database_url": "postgresql://localhost/mydb", "debug": true}')
# Point to config directory
os.environ["PYDANTIC_JSON_PATH"] = "./config"
config = AppConfig()
Configuration Priority
Values are loaded in the following priority order (highest to lowest):
- Environment variables (highest priority)
- Vault KV store
- JSON file (
config.json) - Initialization parameters
- .env file
- Default values (lowest priority)
How It Works
vltconfig extends pydantic-settings to add custom settings sources:
from vltconfig import VaultJsonConfig
class AppConfig(VaultJsonConfig):
# Your fields here
pass
# When you instantiate AppConfig(), it:
# 1. Connects to Vault using configured authentication
# 2. Fetches secrets from the specified path
# 3. Loads config.json if available
# 4. Merges all sources according to priority
# 5. Validates using pydantic
# 6. Returns type-safe configuration object
Migration from vault-pydantic-simple
⚠️ Breaking Change in v1.0.0: The import path has changed.
If you're upgrading from vault-pydantic-simple:
# Uninstall old package
pip uninstall vault-pydantic-simple
# Install new package
pip install vltconfig
Update your imports:
# Old (v0.x)
from vlt import VaultJsonConfig
# New (v1.0.0+)
from vltconfig import VaultJsonConfig
Requirements
- Python >= 3.11
- pydantic >= 2.10.6
- pydantic-settings >= 2.7.1
- hvac >= 2.3.0 (HashiCorp Vault client)
- loguru >= 0.7.3 (logging)
- tenacity >= 9.0.0 (retry logic)
Examples
The repository includes practical examples in the examples/ directory:
📝 Available Examples
-
debug_example.py - Complete debugging setup
- Shows how to enable detailed TRACE logging
- Demonstrates custom JSON config path setup
- Full example with all configuration sources
-
json_only_example.py - JSON-only configuration
- Using vltconfig without Vault (local development)
- Shows graceful fallback when Vault is not configured
- Environment variable override examples
🚀 Running Examples
# Clone the repository
git clone https://github.com/tofuurem/vltconfig.git
cd vltconfig
# Run the debug example
python examples/debug_example.py
# Run the JSON-only example
python examples/json_only_example.py
📂 Example Config Files
The examples/config/ directory contains sample config.json files you can use as templates:
{
"database_url": "postgresql://localhost:5432/mydb",
"api_key": "your-api-key-here",
"debug": true,
"max_connections": 20
}
Development
# Clone repository
git clone https://github.com/tofuurem/vltconfig.git
cd vltconfig
# Install dependencies with uv
uv sync --group dev
# Run tests
uv run pytest --cov=vltconfig
# Run linters
uv run ruff check .
uv run mypy vltconfig/
Testing
# Run all tests
uv run pytest
# Run with coverage
uv run pytest --cov=vltconfig --cov-report=term-missing
# Run specific test file
uv run pytest tests/unit/test_config.py -v
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
License
MIT License - see LICENSE file for details.
Links
- PyPI: https://pypi.org/project/vltconfig/
- GitHub: https://github.com/tofuurem/vltconfig
- Issues: https://github.com/tofuurem/vltconfig/issues
Author
tofuurem - rabbit_1399@icloud.com
Made with ❤️ using Python and Pydantic
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file vltconfig-1.0.1.tar.gz.
File metadata
- Download URL: vltconfig-1.0.1.tar.gz
- Upload date:
- Size: 12.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
df6f1017692cbcf542b96b765127526b1a5df3355c85d5bcd8136ed29188821d
|
|
| MD5 |
a7a4541f700367675e89602775a84a22
|
|
| BLAKE2b-256 |
a311b7b2a5e813d69b9e286340ab2a19315ad6fa47bcb7979991a8d8bbacb842
|
Provenance
The following attestation bundles were made for vltconfig-1.0.1.tar.gz:
Publisher:
main.yml on tofuurem/vltconfig
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
vltconfig-1.0.1.tar.gz -
Subject digest:
df6f1017692cbcf542b96b765127526b1a5df3355c85d5bcd8136ed29188821d - Sigstore transparency entry: 630294327
- Sigstore integration time:
-
Permalink:
tofuurem/vltconfig@edbcb2f3e30b71c8c6f74cf4611648f71ba5c639 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/tofuurem
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
main.yml@edbcb2f3e30b71c8c6f74cf4611648f71ba5c639 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file vltconfig-1.0.1-py3-none-any.whl.
File metadata
- Download URL: vltconfig-1.0.1-py3-none-any.whl
- Upload date:
- Size: 10.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e2d4d84875f8459541f98ea5874705c78318908d4ffdb97804aaac07492566f1
|
|
| MD5 |
796e27f341dddc5d7d9e1308cb2c52a9
|
|
| BLAKE2b-256 |
b865a899ecac1a219a1529d9d6193d61c194604365b73f746c85edb7f00dff17
|
Provenance
The following attestation bundles were made for vltconfig-1.0.1-py3-none-any.whl:
Publisher:
main.yml on tofuurem/vltconfig
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
vltconfig-1.0.1-py3-none-any.whl -
Subject digest:
e2d4d84875f8459541f98ea5874705c78318908d4ffdb97804aaac07492566f1 - Sigstore transparency entry: 630294333
- Sigstore integration time:
-
Permalink:
tofuurem/vltconfig@edbcb2f3e30b71c8c6f74cf4611648f71ba5c639 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/tofuurem
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
main.yml@edbcb2f3e30b71c8c6f74cf4611648f71ba5c639 -
Trigger Event:
workflow_dispatch
-
Statement type: