Profile Config

Profile-based configuration management for Python applications.

What It Does

Profile Config manages application configuration using profiles (e.g., development, staging, production). It discovers configuration files in your project hierarchy, merges them with proper precedence, and resolves the requested profile.

Configuration Flow

1. Discovery Phase
   Search locations (highest to lowest precedence):
   ./myapp/config.yaml          <- Current directory
   ../myapp/config.yaml         <- Parent directory
   ../../myapp/config.yaml      <- Grandparent directory
   ~/myapp/config.yaml         <- Home directory

2. Merge Phase
   Files are merged with closer files taking precedence

3. Profile Resolution
   defaults + profile + inherited profiles

4. Override Phase
   Apply runtime overrides (highest precedence)

5. Interpolation Phase
   Resolve ${variable} references

Example

Given this configuration file at myapp/config.yaml:

defaults:
  host: localhost
  port: 5432
  debug: false

profiles:
  development:
    debug: true
    database: myapp_dev

  production:
    host: prod-db.example.com
    database: myapp_prod

This code:

from profile_config import ProfileConfigResolver

resolver = ProfileConfigResolver("myapp", profile="development")
config = resolver.resolve()

Produces this configuration:

{
    "host": "localhost",      # from defaults
    "port": 5432,             # from defaults
    "debug": True,            # from development profile (overrides defaults)
    "database": "myapp_dev"   # from development profile
}

Installation

pip install profile-config

For TOML support on Python < 3.11:

pip install profile-config[toml]

Basic Usage

1. Create Configuration File

Create myapp/config.yaml in your project:

defaults:
  timeout: 30
  retries: 3

profiles:
  development:
    debug: true
    log_level: DEBUG

  production:
    debug: false
    log_level: WARNING

2. Load Configuration

from profile_config import ProfileConfigResolver

# Load development profile
resolver = ProfileConfigResolver("myapp", profile="development")
config = resolver.resolve()

# Access configuration values
print(config["timeout"])    # 30 (from defaults)
print(config["debug"])      # True (from development profile)
print(config["log_level"])  # DEBUG (from development profile)

Configuration File Discovery

Profile Config searches for configuration files by walking up the directory tree from the current working directory, then checking the home directory.

Search Pattern

Default pattern: {config_name}/{profile_filename}.{extension}

Examples:

• myapp/config.yaml (default)
• myapp/settings.yaml (custom filename)
• myapp/app.json (custom filename)

Search Order (highest to lowest precedence)

Current directory:     ./myapp/config.yaml
Parent directory:      ../myapp/config.yaml
Grandparent directory: ../../myapp/config.yaml
...
Home directory:        ~/myapp/config.yaml

File Extensions

Searches for files with these extensions (in order):

• .yaml
• .yml
• .json
• .toml

Example Directory Structure

/home/user/projects/myapp/
├── backend/
│   └── myapp/
│       └── config.yaml      <- Project-specific config
└── myapp/
    └── config.yaml          <- Shared config

/home/user/myapp/
└── config.yaml              <- User-specific config

When running from /home/user/projects/myapp/backend/:

1. Finds ./myapp/config.yaml (current directory)
2. Finds ../myapp/config.yaml (parent directory)
3. Finds ~/myapp/config.yaml (home directory)
4. Merges all three (current directory has highest precedence)

Custom Profile Filename

Use a different filename instead of config:

# Search for settings.yaml instead of config.yaml
resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    profile_filename="settings"
)

This searches for:

• ./myapp/settings.yaml
• ../myapp/settings.yaml
• ~/myapp/settings.yaml

Use cases:

• Organization naming standards (e.g., settings.yaml)
• Multiple configuration types in same directory
• Legacy system compatibility
• More descriptive names (e.g., database.yaml, api.yaml)

Configuration File Format

Structure

# Optional: specify which profile to use by default
default_profile: development

# Optional: values applied to all profiles
defaults:
  timeout: 30
  retries: 3

# Required: profile definitions
profiles:
  development:
    debug: true
    database: myapp_dev

  production:
    debug: false
    database: myapp_prod

Supported Formats

YAML

defaults:
  host: localhost
  port: 5432

profiles:
  development:
    debug: true

JSON

{
  "defaults": {
    "host": "localhost",
    "port": 5432
  },
  "profiles": {
    "development": {
      "debug": true
    }
  }
}

TOML

[defaults]
host = "localhost"
port = 5432

[profiles.development]
debug = true

Using the Default Profile

The "default" profile has special behavior that makes it easy to use only the defaults section without defining an explicit profile.

Automatic Creation

When you request profile="default" and no "default" profile exists in your configuration, an empty profile is automatically created. This returns only the values from the defaults section.

defaults:
  host: localhost
  port: 5432
  timeout: 30

profiles:
  development:
    debug: true
    port: 3000

  production:
    host: prod-db.com
    debug: false

# No explicit "default" profile defined in config
resolver = ProfileConfigResolver("myapp", profile="default")
config = resolver.resolve()

# Returns only defaults:
# {
#     "host": "localhost",
#     "port": 5432,
#     "timeout": 30
# }

Explicit Default Profile

If you define an explicit "default" profile, it takes precedence over the auto-creation:

defaults:
  host: localhost
  port: 5432
  timeout: 30

profiles:
  default:
    timeout: 60      # Override default timeout
    custom: true     # Add custom setting

  development:
    debug: true

resolver = ProfileConfigResolver("myapp", profile="default")
config = resolver.resolve()

# Returns defaults + default profile:
# {
#     "host": "localhost",
#     "port": 5432,
#     "timeout": 60,      # Overridden by default profile
#     "custom": True      # Added by default profile
# }

Use Cases

1. Base configuration without environment-specific overrides:

# Get base configuration only
resolver = ProfileConfigResolver("myapp", profile="default")
base_config = resolver.resolve()

2. Fallback when no specific profile is needed:

import os

# Use environment-specific profile if set, otherwise use defaults
env = os.environ.get("ENV", "default")
resolver = ProfileConfigResolver("myapp", profile=env)
config = resolver.resolve()

3. Testing with minimal configuration:

# Tests can use "default" profile for baseline behavior
def test_app_with_defaults():
    resolver = ProfileConfigResolver("myapp", profile="default")
    config = resolver.resolve()
    # Test with minimal config

Profile Inheritance

Profiles can inherit from other profiles using the inherits key.

Example

profiles:
  base:
    host: localhost
    timeout: 30

  development:
    inherits: base
    debug: true
    database: myapp_dev

  staging:
    inherits: development
    debug: false
    host: staging-db.example.com

Resolution Order

For profile staging:

1. Start with base profile
2. Merge development profile (overrides base)
3. Merge staging profile (overrides development)

Result:

{
    "host": "staging-db.example.com",  # from staging (overrides base)
    "timeout": 30,                      # from base
    "debug": False,                     # from staging (overrides development)
    "database": "myapp_dev"             # from development
}

Multi-Level Inheritance

profiles:
  base:
    timeout: 30

  development:
    inherits: base
    debug: true

  development-team1:
    inherits: development
    team_id: team1

  development-team2:
    inherits: development
    team_id: team2

Circular inheritance is detected and raises an error.

Variable Interpolation

Use ${variable} syntax to reference other configuration values.

Example

defaults:
  app_name: myapp
  base_path: /opt/${app_name}
  data_path: ${base_path}/data
  log_path: ${base_path}/logs

profiles:
  development:
    base_path: /tmp/${app_name}

Resolution

For profile development:

{
    "app_name": "myapp",
    "base_path": "/tmp/myapp",           # interpolated
    "data_path": "/tmp/myapp/data",      # interpolated
    "log_path": "/tmp/myapp/logs"        # interpolated
}

## Environment Variables

You can automatically set environment variables from your configuration using the `env_vars` section. This feature is useful for applications that read configuration from `os.environ` or for setting up the environment before importing modules.

### Basic Usage

```yaml
defaults:
  env_vars:
    DATABASE_URL: "postgresql://localhost:5432/mydb"
    LOG_LEVEL: "INFO"
    API_TIMEOUT: "30"

profiles:
  production:
    env_vars:
      DATABASE_URL: "postgresql://prod-db.example.com:5432/mydb"
      LOG_LEVEL: "WARNING"

import os
from profile_config import ProfileConfigResolver

resolver = ProfileConfigResolver("myapp", profile="production")
config = resolver.resolve()

# Environment variables are now set
print(os.environ["DATABASE_URL"])  # postgresql://prod-db.example.com:5432/mydb
print(os.environ["LOG_LEVEL"])      # WARNING
print(os.environ["API_TIMEOUT"])    # 30

The env_vars section is automatically removed from the returned configuration.

Variable Interpolation

Environment variables support interpolation from other configuration values:

defaults:
  app_name: myapp
  database:
    host: localhost
    port: 5432
    name: mydb
  env_vars:
    APP_NAME: "${app_name}"
    DATABASE_URL: "postgresql://${database.host}:${database.port}/${database.name}"
    DATA_DIR: "/var/data/${app_name}"

profiles:
  production:
    database:
      host: prod-db.example.com

resolver = ProfileConfigResolver("myapp", profile="production")
config = resolver.resolve()

print(os.environ["DATABASE_URL"])  # postgresql://prod-db.example.com:5432/mydb

Profile-Specific Environment Variables

Environment variables merge with profile overrides:

defaults:
  env_vars:
    LOG_LEVEL: "INFO"
    DEBUG_MODE: "false"

profiles:
  development:
    env_vars:
      LOG_LEVEL: "DEBUG"
      DEBUG_MODE: "true"
      DEV_SERVER: "http://localhost:8000"

  production:
    env_vars:
      LOG_LEVEL: "WARNING"
      SENTRY_DSN: "https://..."

Safe Mode (Default)

By default, existing environment variables are not overwritten. This respects values injected by container orchestration (Docker, Kubernetes) or CI/CD systems:

import os

# Variable already exists
os.environ["LOG_LEVEL"] = "ERROR"

resolver = ProfileConfigResolver("myapp", profile="development")
config = resolver.resolve()

# LOG_LEVEL remains "ERROR" (not overwritten)
print(os.environ["LOG_LEVEL"])  # ERROR

# Check what was skipped
env_info = resolver.get_environment_info()
print(env_info["skipped"])  # {'LOG_LEVEL': 'DEBUG'}

Override Mode

Force configuration values to override existing environment variables:

resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    override_environment=True  # Override existing vars
)
config = resolver.resolve()

# LOG_LEVEL is now overwritten with config value

Disable Environment Variables

Disable the feature entirely:

resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    apply_environment=False  # Don't set any environment variables
)
config = resolver.resolve()

# Environment variables are not set
# env_vars section remains in returned config

Custom Key Name

Use a different key name instead of env_vars:

defaults:
  exports:  # Custom key name
    MY_VAR: "value"

resolver = ProfileConfigResolver(
    "myapp",
    environment_key="exports"  # Use 'exports' instead of 'env_vars'
)
config = resolver.resolve()

Type Conversion

Non-string values are automatically converted to strings:

defaults:
  env_vars:
    PORT: 8080        # Integer
    DEBUG: true       # Boolean
    RATIO: 3.14       # Float
    EMPTY: null       # Null

Results in:

os.environ["PORT"]   # "8080"
os.environ["DEBUG"]  # "True"
os.environ["RATIO"]  # "3.14"
os.environ["EMPTY"]  # ""

Tracking Applied Variables

Check which environment variables were applied or skipped:

resolver = ProfileConfigResolver("myapp", profile="development")
config = resolver.resolve()

env_info = resolver.get_environment_info()

# Variables that were set
for key, value in env_info["applied"].items():
    print(f"Set {key}={value}")

# Variables that were skipped (already existed)
for key, value in env_info["skipped"].items():
    print(f"Skipped {key} (config wanted '{value}', kept existing value)")

Configuration Options

ProfileConfigResolver(
    config_name="myapp",
    profile="development",
    apply_environment=True,          # Enable/disable feature (default: True)
    environment_key="env_vars",      # Config key name (default: "env_vars")
    override_environment=False,      # Override existing vars (default: False)
)

Use Cases

Application Bootstrap: Set environment variables before importing modules that read from os.environ:

# Bootstrap environment first
resolver = ProfileConfigResolver("myapp", profile="production")
config = resolver.resolve()

# Now safe to import modules that use os.environ
import my_app
my_app.run()

Container Orchestration: Provide defaults while respecting injected secrets:

# Kubernetes injects secrets as environment variables
# Config provides non-sensitive defaults
# override_environment=False ensures secrets aren't overwritten
resolver = ProfileConfigResolver(
    "myapp",
    profile="production",
    override_environment=False  # Respect K8s secrets
)

Development Environment: Override everything for local development:

# Force all config values for consistent dev environment
resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    override_environment=True  # Override everything
)

Security Considerations

• Never commit sensitive values (passwords, API keys) to configuration files
• Use secret management systems (Kubernetes Secrets, AWS Secrets Manager, etc.) for credentials
• The env_vars feature is for non-sensitive configuration values
• Default behavior (override_environment=False) is safe - respects externally injected secrets

Variables are resolved after profile inheritance is complete.

## Runtime Overrides

Apply configuration overrides at runtime with highest precedence.

### Dictionary Override

```python
resolver = ProfileConfigResolver(
    "myapp",
    profile="production",
    overrides={"debug": True, "host": "test-db.example.com"}
)
config = resolver.resolve()

File Override

resolver = ProfileConfigResolver(
    "myapp",
    profile="production",
    overrides="/path/to/overrides.yaml"
)
config = resolver.resolve()

Supported formats: .yaml, .yml, .json, .toml

List of Overrides

Apply multiple overrides in order (later overrides take precedence):

resolver = ProfileConfigResolver(
    "myapp",
    profile="production",
    overrides=[
        "/path/to/base-overrides.yaml",
        {"debug": True},
        "/path/to/final-overrides.json"
    ]
)
config = resolver.resolve()

Precedence Order

1. Discovered config files (lowest)
2. Profile resolution
3. Override 1
4. Override 2
5. Override N (highest)

Configuration Options

Customize Search Behavior

resolver = ProfileConfigResolver(
    config_name="myapp",
    profile="development",
    extensions=["yaml", "json"],  # Only search these formats
    search_home=False,            # Don't search home directory
)

Custom Profile Filename

# Use settings.yaml instead of config.yaml
resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    profile_filename="settings"
)

Custom Inheritance Key

# Use 'extends' instead of 'inherits'
resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    inherit_key="extends"
)

Disable Variable Interpolation

resolver = ProfileConfigResolver(
    "myapp",
    profile="development",
    enable_interpolation=False
)

Utility Methods

List Available Profiles

resolver = ProfileConfigResolver("myapp")
profiles = resolver.list_profiles()
print(profiles)  # ['development', 'staging', 'production']

Get Discovered Files

resolver = ProfileConfigResolver("myapp")
files = resolver.get_config_files()
for file_path in files:
    print(file_path)

Error Handling

Profile Config raises specific exceptions for different error conditions.

Exception Types

from profile_config.exceptions import (
    ConfigNotFoundError,      # No configuration files found
    ProfileNotFoundError,     # Requested profile doesn't exist
    CircularInheritanceError, # Circular inheritance detected
    ConfigFormatError         # Invalid configuration file format
)

Example

from profile_config import ProfileConfigResolver
from profile_config.exceptions import ProfileNotFoundError

try:
    resolver = ProfileConfigResolver("myapp", profile="nonexistent")
    config = resolver.resolve()
except ProfileNotFoundError as e:
    print(f"Profile not found: {e}")
    # Handle error (use default profile, exit, etc.)

Common Patterns

Environment-Based Configuration

import os
from profile_config import ProfileConfigResolver

env = os.environ.get("ENV", "development")
resolver = ProfileConfigResolver("myapp", profile=env)
config = resolver.resolve()

Team-Specific Configuration

profiles:
  development:
    debug: true

  development-team1:
    inherits: development
    team_id: team1
    endpoint: "https://team1.internal.com"

  development-team2:
    inherits: development
    team_id: team2
    endpoint: "https://team2.internal.com"

import os
from profile_config import ProfileConfigResolver

team = os.environ.get("TEAM_NAME", "")
env = os.environ.get("ENV", "development")
profile = f"{env}-{team}" if team else env

resolver = ProfileConfigResolver("myapp", profile=profile)
config = resolver.resolve()

Configuration with Secrets

Store secrets separately and merge at runtime:

import json
from pathlib import Path
from profile_config import ProfileConfigResolver

# Load base configuration
resolver = ProfileConfigResolver("myapp", profile="production")
config = resolver.resolve()

# Load secrets from secure location
secrets_file = Path("/etc/secrets/myapp.json")
if secrets_file.exists():
    with open(secrets_file) as f:
        secrets = json.load(f)
    config.update(secrets)

Or use overrides:

resolver = ProfileConfigResolver(
    "myapp",
    profile="production",
    overrides="/etc/secrets/myapp.json"
)
config = resolver.resolve()

Format Comparison

When to Use Each Format

YAML: Complex nested configurations, human-edited files JSON: API integration, machine-generated configs, data exchange TOML: Application configuration with type safety, flat structures

Examples

The examples/ directory contains working examples:

• basic_usage.py - Basic configuration and profile usage
• advanced_profiles.py - Inheritance patterns and error handling
• default_profile_usage.py - Default profile auto-creation and use cases
• web_app_config.py - Web application configuration management
• env_vars_example.py - Environment variable injection and configuration
• toml_usage.py - TOML format features

Run examples:

cd examples
python basic_usage.py

Development

Setup

git clone https://github.com/bassmanitram/profile-config.git
cd profile-config
pip install -e ".[dev,toml]"

Run Tests

pytest

Run Tests with Coverage

pytest --cov=profile_config --cov-report=html

API Reference

ProfileConfigResolver

ProfileConfigResolver(
    config_name: str,
    profile: str = "default",
    profile_filename: str = "config",
    overrides: Optional[Union[Dict, PathLike, List[Union[Dict, PathLike]]]] = None,
    extensions: Optional[List[str]] = None,
    search_home: bool = True,
    inherit_key: str = "inherits",
    enable_interpolation: bool = True,
    apply_environment: bool = True,
    environment_key: str = "env_vars",
    override_environment: bool = False,
)

Parameters:

• config_name: Name of configuration directory (e.g., "myapp")
• profile: Profile name to resolve (default: "default")
• profile_filename: Name of profile file without extension (default: "config")
• overrides: Override values (dict, file path, or list of dicts/paths)
• extensions: File extensions to search (default: ["yaml", "yml", "json", "toml"])
• search_home: Whether to search home directory (default: True)
• inherit_key: Key name for profile inheritance (default: "inherits")
• enable_interpolation: Whether to enable variable interpolation (default: True)
• apply_environment: Whether to apply environment variables from config (default: True)
• environment_key: Key name for environment variables section (default: "env_vars")
• override_environment: Whether to override existing environment variables (default: False)

Methods:

• resolve() -> Dict[str, Any]: Resolve and return configuration
• list_profiles() -> List[str]: List available profiles
• get_config_files() -> List[Path]: Get discovered configuration files
• get_environment_info() -> Dict[str, Dict[str, str]]: Get information about applied/skipped environment variables
• get_config_files() -> List[Path]: Get discovered configuration files

License

MIT License. See LICENSE file for details.

Contributing

Contributions are welcome. Please read CONTRIBUTING.md for guidelines.

Links

• GitHub: https://github.com/bassmanitram/profile-config
• PyPI: https://pypi.org/project/profile-config/
• Issues: https://github.com/bassmanitram/profile-config/issues

Changelog

See CHANGELOG.md for version history.