yaml-arg-configer

A Python library for combining YAML configuration files with command-line arguments. It provides a flexible configuration management system with support for multiple config files, context variables, module imports, and function calls.

Table of Contents

• Installation
• Overview
• Quick Start
• Configuration Structure
• Custom YAML Tags
• Command-Line Options
• Documentation
• Troubleshooting
• License

Installation

# Install from PyPI (when available)
pip install yaml-arg-configer

# Install from source (editable)
pip install -e .

Overview

The YamlArgParser class bridges YAML configuration files and command-line arguments. It:

1. Loads default values from a YAML file
2. Merges with user-provided configurations
3. Applies command-line overrides
4. Supports dynamic value resolution via custom YAML tags

Features

Quick Start

Basic Example

from yaml_arg_configer import YamlArgParser

# Initialize parser with default YAML file
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse command-line arguments
cmd_args = yaml_parser.parse_cmd_args()

# Parse configurations from YAML files
final_args = yaml_parser.parse_args(cmd_args)

# Access results
print(yaml_parser.get_args())           # argparse.Namespace
print(yaml_parser.args)                 # dict
print(yaml_parser['arg_name'])           # dict-like access
print(yaml_parser.get('arg_name'))       # dict-like access with default value, return None by default if arg not exist
yaml_parser.save_config('output.yaml')   # Save to YAML

Programmatic Usage

from yaml_arg_configer import YamlArgParser

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse defaults
yaml_parser.parse_default('defaults.yaml')

# Update from additional YAML files
yaml_parser.update_from_yaml('config1.yaml', strict=True)
yaml_parser.update_from_yaml('config2.yaml', strict=True)

# Get final arguments as dict
final_args = yaml_parser.args

# Or as argparse.Namespace
final_args = yaml_parser.get_args()

Saving Configuration

# Save current configuration to YAML
yaml_parser.save_config('output_config.yaml')

Parsing Configurations from Directory

The parse_cfgs() method is convenient for loading configurations from a directory:

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse configurations from files in a directory
# Loads: config1.yaml, config2.yaml, etc.
final_args = yaml_parser.parse_cfgs(
    cfgs=['config1', 'config2'],  # Config file names (without .yaml)
    cfg_dir='configs',             # Directory containing config files
    strict=True                    # Enforce validation
)

# Access results
print(final_args)  # Returns dict with parsed arguments

Key Features:

• Automatically constructs file paths: {cfg_dir}/{name}.yaml
• Loads context from ctx.yaml if specified
• Parses defaults before applying config files
• Returns merged configuration as dictionary

Configuration Structure

Default Configuration (defaults.yaml)

arg1:
  default: ~                    # ~ means None
  help: "Description for arg1"
arg2:
  default: 42
  help: "Number of iterations"
arg3: True                      # Simple format: value directly
arg4: "string value"            # Simple format: string value

User Configuration (config1.yaml)

arg1: value1                    # Override default value
arg2: 100                       # Override another value

Context Management

Context YAML files support dynamic context management:

# ctx.yaml
import: !ipt
  torch.nn:
    relu: ReLU
    gelu: GELU

The !ipt tag can import multiple modules and attributes, which are then available for !ctx references.

Custom YAML Tags

The library extends PyYAML with custom tags for dynamic value resolution: !ctx (context variables), !ipt (import module attributes), !call (function calls), !join (flatten lists), !chain2path (join paths).

Full reference with examples: docs/yaml-tags.md

Command-Line Options

Example Commands

# Load defaults and config files
python examples/my_script.py --dc defaults.yaml -d configs -c config1 config2

# With context resolution
python examples/my_script.py --dc defaults.yaml -ctx ctx.yaml -d configs -c config1

# Get help documentation
python examples/my_script.py --docs
python examples/my_script.py --doc arg1 arg2

# Strict mode (enforce config validation)
python examples/my_script.py --dc defaults.yaml --strict -d configs -c config1

Documentation

• API Reference — full Python API for YamlArgParser and the YAML loader
• YAML Tags — custom tag reference with examples

Troubleshooting

Best Practices

1. Start with defaults: Define comprehensive defaults in defaults.yaml
2. Use context files: Centralize reusable values in ctx.yaml
3. Document with help: Always provide help text for arguments
4. Validate with strict: Use --strict to catch configuration errors
5. Chain configs: Load multiple configs for modular configuration

Common Issues

Missing keys in strict mode

• Symptom: AssertionError: Argument 'X' not defined in the default configuration.
• Solution: Ensure all user config keys appear in defaults.yaml with proper structure (even if using minimal entries)

Context variables not resolving

• Symptom: KeyError: xxx when loading the config
• Check that ctx.yaml exists and is loaded via -ctx flag
• Verify import paths are correct in context yaml (import: !ipt)
• Variable names match between import and reference

Config merging order wrong

• Symptom: Config file values overwritten by defaults instead of overriding them
• Explanation: Priority order is: CLI args > user configs > defaults. Ensure config files load after defaults

License

MIT