dynamic-config-loader 1.0.2

A lightweight, strategy-driven engine to dynamically discover and recursively deep-merge layered YAML configurations.

Dynamic Configuration Loader

A lightweight, production-grade, strategy-driven configuration orchestration engine designed to dynamically discover, alphabetically sort, and recursively deep-merge multi-layered YAML configurations across variable filesystem paths.

Copyright (C) 2026 Avinash Kumar <halfhiddencode AT gmail DOT com>

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

Table of Contents

1. Core Features

2. Repository Link

3. Installation

4. Quick Start & Basic Usage

   • 1. Define Architecture
   • 2. Execute Loader Code
   • 3. Expected Output

5. Advanced Production Use Cases

   • Use Case A: Kubernetes Pod Deployment Sidecars
   • Use Case B: Multi-Cloud Region Failover Schemes
   • Use Case C: Microservice Feature Flag Isolation

6. Production Security & Logging Architecture

   • Secure Mode (INFO)
   • Diagnostic Mode (DEBUG)

7. Automated Unit Testing

8. Contributing

---

Core Features

• Layered Ingestion Architecture: Seeds a master configuration matrix from a static baseline file and patches runtime override layers over it dynamically.
• Dynamic Multi-Directory Scanning: Discovers and processes external asset folders via an isolated environment variable vector (PYTHON_ADDITIONAL_CONFIG).
• Deterministic Processing Loop: Enforces explicit alphabetical sorting on all discovered override files to make overlapping configuration mutations completely predictable.
• Recursive Deep Merges: Protects nested dictionary tracking structures from structural wiping or collision drops during runtime updates.
• Dual-Layered Operational Security Logs: Structural actions log cleanly at INFO with sensitive values redacted. Toggling to DEBUG unlocks full value visibility alongside a prominent terminal security warning banner.
• Zero Bulk Dependencies: Relies entirely on native Python system standards and pyyaml.

---

Repository Link

The official source code, distribution assets, and issue tracking boards are hosted on GitLab: python-dynamic-config-loader

---

Installation

You can install the package directly from PyPI using standard package installers:

pip install dynamic-config-loader

Alternatively, if you are developing locally or installing from a private distribution wheels repository:

pip install .

---

Quick Start & Basic Usage

1. Define Your Configuration Architecture

Set up your files to match this example structural multi-layer layout:

Baseline File: test_configs/base_config.yaml

app_name: "Production Gateway"
debug_mode: false
database:
  host: "127.0.0.1"
  port: 5432
  pool_size: 20

Override Patch File: test_configs/overrides/01_dev_patch.yaml

debug_mode: true
database:
  host: "internal-dev-cluster.local"

2. Execute the Loader Code

Import the engine into your main execution entry point script (e.g., run.py):

import os
import json
import logging
from dynamic_config_loader import DynamicConfigLoader

# 1. (Optional) Configure logging to view the configuration orchestration sequence
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(name)s: %(message)s")

# 2. Point the system context env variable to your overrides directory/directories
os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./test_configs/overrides"

# 3. Instantiate the loading engine with your baseline tracking path
loader = DynamicConfigLoader(default_config_path="./test_configs/base_config.yaml")

# 4. Trigger the ingestion engine pipeline loop
final_config = loader.load()

# 5. Access your compiled matrix data properties safely
print(json.dumps(final_config, indent=4))

3. Expected Output Matrix

{
    "app_name": "Production Gateway",
    "debug_mode": true,
    "database": {
        "host": "internal-dev-cluster.local",
        "port": 5432,
        "pool_size": 20
    }
}

---

Advanced Production Use Cases

This section illustrates how to import and consume the library across different enterprise deployment patterns.

Use Case A: Kubernetes Pod Deployment Sidecars

In a Kubernetes infrastructure, you often want a shared immutable base image, but need to patch runtime properties dynamically via external ConfigMaps mounted inside the container filesystem.

import os
from dynamic_config_loader import DynamicConfigLoader

def run_kubernetes_runtime_boot():
    # Base configuration is packaged inside the immutable Docker image layer
    BASE_PATH = "/app/core/configs/base_app_config.yaml"
    
    # Mount point targets for external Kubernetes ConfigMap volumes mapped dynamically
    # Example: 01_resources_patch.yaml, 02_autoscaling_tweaks.yaml
    os.environ["PYTHON_ADDITIONAL_CONFIG"] = "/etc/pod-config-mounts/overrides"
    
    loader = DynamicConfigLoader(default_config_path=BASE_PATH)
    runtime_config = loader.load()
    
    return runtime_config

Use Case B: Multi-Cloud Region Failover Schemes

When deploying an application across multiple cloud environments (e.g., AWS and Azure) or geographically isolated zones, you can chain multiple override lookup paths using comma separation.

import os
from dynamic_config_loader import DynamicConfigLoader

def initialize_multi_cloud_mesh():
    # Baseline defaults common across all global infrastructure regions
    common_base = "./infrastructure/global_defaults.yaml"
    
    # Cascade configuration priorities sequentially by tokenizing directories:
    # 1. Layer cloud-specific parameters (Azure vs AWS)
    # 2. Layer region-specific endpoints (East US vs West US)
    os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./infrastructure/cloud/azure, ./infrastructure/regions/eastus"
    
    loader = DynamicConfigLoader(default_config_path=common_base)
    
    # The engine walks through both directories alphabetically, executing a safe recursive deep-merge
    active_cloud_matrix = loader.load()
    return active_cloud_matrix

Use Case C: Microservice Feature Flag Isolation

For testing out modern canary releases or runtime flags without updating base deployment states, you can point the engine to a scratch or ephemeral folder containing experimental overrides.

import os
from dynamic_config_loader import DynamicConfigLoader

class MicroserviceOrchestrator:
    def __init__(self):
        self.base_config_path = "./services/gateway/manifest.yaml"
        
    def compile_active_manifest(self, isolated_canary_dir: str = None):
        if isolated_canary_dir:
            # Dynamically point the system environmental variable vector to an ephemeral directory
            os.environ["PYTHON_ADDITIONAL_CONFIG"] = isolated_canary_dir
        else:
            # Fall back to default production path states
            os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./services/gateway/production_overrides"
            
        loader = DynamicConfigLoader(default_config_path=self.base_config_path)
        return loader.load()

---

Production Security & Logging Architecture

The engine features isolated logging via the dynamic_config_loader log domain. It respects your production secrets by modifying output profiles based on the standard Python severity thresholds:

Secure Production Mode (logging.INFO)

By default, the loader protects credentials, tokens, and passwords from leaking into system scrapers or monitoring logs. It reports structural pipeline keys but keeps their underlying values redacted:

2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Ingesting baseline configuration layer from target source: ./test_configs/base_config.yaml
2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Processing override layer discovery file: 01_dev_patch.yaml
2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Updating configuration value at path 'database.host' [Value Redacted]

Diagnostic Troubleshooting Mode (logging.DEBUG)

If you need full visibility to debug a complex deep-merge mutation sequence, drop the log module level down to DEBUG. The loader will output an explicit security notice warning exactly once, followed by the raw values:

2026-06-06 12:35:15 [WARNING] dynamic_config_loader.loader: 
=======================================================================
SECURITY NOTICE: DYNAMIC CONFIG LOADER OPERATING IN VERBOSE DEBUG MODE
Execution traces will output raw parameter contents to the log stream.
Ensure this terminal session does not capture or leak sensitive secrets.
=======================================================================

2026-06-06 12:35:15 [DEBUG] dynamic_config_loader.loader: Updating configuration value at path 'database.host': '127.0.0.1' -> 'internal-dev-cluster.local'

To selectively control or mute the configuration pipeline logging without affecting your main application logs, configure the module logger explicitly:

logging.getLogger("dynamic_config_loader").setLevel(logging.WARNING)

Contributing

We welcome additions, optimization rewrites, and security updates! Please thoroughly review our guidelines inside CONTRIBUTING.md to understand local sandbox workspace provisioning, testing workflows, and branching protocols before filing a Pull Request.