yaml-arg-configer

A Python library for combining YAML configuration files with command-line arguments. It provides a flexible configuration management system with support for multiple config files, context variables, module imports, and function calls.

Table of Contents

• Installation
• Overview
• Quick Start
• Configuration Structure
• Custom YAML Tags
• Command-Line Options
• Documentation
• Troubleshooting
• License

Installation

# Install from PyPI (when available)
pip install yaml-arg-configer

# Install from source (editable)
pip install -e .

Overview

The YamlArgParser class bridges YAML configuration files and command-line arguments. It:

1. Loads default values from a YAML file
2. Merges with user-provided configurations
3. Selects which config files to load via the command line
4. Supports dynamic value resolution via custom YAML tags

Scope (by design):

• Flat key=value configs: each YAML key is one parameter. No deep merge, no dot-keys — nested values are used as-is, not overridden per sub-key.
• CLI selects files, YAML sets values: config keys appear in --help with their help text, but values always come from config files, never the command line.
• Hydra-lite: PyYAML-only, deliberately not growing into Hydra-style nested configs / config groups.

Features

Quick Start

Basic Example

from yaml_arg_configer import YamlArgParser

# Initialize parser with default YAML file
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse command-line arguments
cmd_args = yaml_parser.parse_cmd_args()

# Parse configurations from YAML files
final_args = yaml_parser.parse_args(cmd_args)

# Access results
print(yaml_parser.get_args())           # argparse.Namespace
print(yaml_parser.args)                 # dict
print(yaml_parser['arg_name'])           # dict-like access
print(yaml_parser.get('arg_name'))       # dict-like access with default value, return None by default if arg not exist
yaml_parser.save_config('output.yaml')   # Save to YAML

Programmatic Usage

from yaml_arg_configer import YamlArgParser

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse defaults
yaml_parser.parse_default('defaults.yaml')

# Update from additional YAML files
yaml_parser.update_from_yaml('config1.yaml', strict=True)
yaml_parser.update_from_yaml('config2.yaml', strict=True)

# Get final arguments as dict
final_args = yaml_parser.args

# Or as argparse.Namespace
final_args = yaml_parser.get_args()

Saving Configuration

# Save current configuration to YAML
yaml_parser.save_config('output_config.yaml')

Parsing Configurations from Directory

The parse_cfgs() method is convenient for loading configurations from a directory:

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse configurations from files in a directory
# Loads: config1.yaml, config2.yaml, etc.
final_args = yaml_parser.parse_cfgs(
    cfgs=['config1', 'config2'],  # Config file names (without .yaml)
    cfg_dir='configs',             # Directory containing config files
    strict=True                    # Enforce validation
)

# Access results
print(final_args)  # Returns dict with parsed arguments

Key Features:

• Automatically constructs file paths: {cfg_dir}/{name}.yaml
• Loads context from ctx.yaml if specified
• Parses defaults before applying config files
• Returns merged configuration as dictionary

Configuration Structure

Default Configuration (defaults.yaml)

arg1:
  default: ~                    # ~ means None
  help: "Description for arg1"
arg2:
  default: 42
  help: "Number of iterations"
arg3: True                      # Simple format: value directly
arg4: "string value"            # Simple format: string value

User Configuration (config1.yaml)

arg1: value1                    # Override default value
arg2: 100                       # Override another value

Context Management

Context YAML files support dynamic context management:

# ctx.yaml
import: !ipt
  math: [sin, cos]

The !ipt tag can import multiple modules and attributes, which are then available for !ctx references.

Custom YAML Tags

The library extends PyYAML with custom tags for dynamic value resolution: !ctx (context variables), !ipt (import module attributes), !call (function calls), !join (flatten lists), !chain2path (join paths).

Full reference with examples: docs/yaml-tags.md

Command-Line Options

Config keys are registered as help-only --<key> options: they show up in --help, but passing a value on the command line errors out — values come from config files. Pass register_help_keys=False to the constructor to disable this registration.

Example Commands

# Load defaults and config files
python examples/my_script.py --dc defaults.yaml -d configs -c config1 config2

# With context resolution
python examples/my_script.py --dc defaults.yaml -ctx ctx.yaml -d configs -c config1

# Show help (built-in flags + all config keys)
python examples/my_script.py --dc defaults.yaml --help

# Strict mode (enforce config validation)
python examples/my_script.py --dc defaults.yaml --strict -d configs -c config1

Documentation

• API Reference — full Python API for YamlArgParser and the YAML loader
• YAML Tags — custom tag reference with examples

Troubleshooting

Best Practices

1. Start with defaults: Define comprehensive defaults in defaults.yaml
2. Use context files: Centralize reusable values in ctx.yaml
3. Document with help: Always provide help text for arguments
4. Validate with strict: Use --strict to catch configuration errors
5. Chain configs: Load multiple configs for modular configuration

Common Issues

Missing keys in strict mode

• Symptom: AssertionError: Argument 'X' not defined in the default configuration.
• Solution: Ensure all user config keys appear in defaults.yaml with proper structure (even if using minimal entries)

Context variables not resolving

• Symptom: KeyError: xxx when loading the config
• Check that ctx.yaml exists and is loaded via -ctx flag
• Verify import paths are correct in context yaml (import: !ipt)
• Variable names match between import and reference

Config merging order wrong

• Symptom: Config file values overwritten by defaults instead of overriding them
• Explanation: Priority order is: later-loaded configs > earlier-loaded configs > defaults. The command line only selects which files load; it does not set values.

License

MIT