yaml-arg-configer 0.1.5

Simplifies the process of parsing command-line arguments and configurations from YAML files

yaml-arg-configer

A Python library for combining YAML configuration files with command-line arguments. It provides a flexible configuration management system with support for multiple config files, context variables, module imports, and function calls.

---

Table of Contents

• Installation
• Overview
• Quick Start
• Configuration Structure
• Custom YAML Tags
• Command-Line Options
• Troubleshooting
• License

---

Installation

# Install from PyPI (when available)
pip install yaml-arg-configer

# Install from source
pip install -e .

---

Overview

The YamlArgParser class bridges YAML configuration files and command-line arguments. It:

1. Loads default values from a YAML file
2. Merges with user-provided configurations
3. Applies command-line overrides
4. Supports dynamic value resolution via custom YAML tags

Features

Feature	Description
Default Configuration	Load base settings from YAML files with default and help fields
Command-Line Override	Override settings via CLI arguments (-c, -d, etc.)
Multiple Configurations	Combine multiple YAML config files in priority order
Context Resolution	Reference context variables using !ctx tag
Module Imports	Import attributes from Python modules using !ipt tag
Function Calls	Execute Python functions using !call tag
Path Operations	Use !join and !chain2path for path manipulation
Config Saving	Export current configuration back to YAML

---

Quick Start

Basic Example

from yaml_arg_configer import YamlArgParser

# Initialize parser with default YAML file
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse command-line arguments
cmd_args = yaml_parser.parse_cmd_args()

# Parse configurations from YAML files
final_args = yaml_parser.parse_args(cmd_args)

# Access results
print(yaml_parser.get_args())           # argparse.Namespace
print(yaml_parser.get_args_dict())       # dict
print(yaml_parser['arg_name'])           # dict-like access
print(yaml_parser.get('arg_name'))       # dict-like access with default value, return None by default if arg not exist
yaml_parser.save_config('output.yaml')   # Save to YAML

Programmatic Usage

from yaml_arg_configer import YamlArgParser

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse defaults
yaml_parser.parse_default('defaults.yaml')

# Update from additional YAML files
yaml_parser.update_from_yaml('config1.yaml', strict=True)
yaml_parser.update_from_yaml('config2.yaml', strict=True)

# Get final arguments as dict
final_args = yaml_parser.get_args_dict()

# Or as argparse.Namespace
final_args = yaml_parser.get_args()

Saving Configuration

# Save current configuration to YAML
yaml_parser.save_config('output_config.yaml')

Parsing Configurations from Directory

The parse_cfgs() method is convenient for loading configurations from a directory:

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse configurations from files in a directory
# Loads: config1.yaml, config2.yaml, etc.
final_args = yaml_parser.parse_cfgs(
    cfgs=['config1', 'config2'],  # Config file names (without .yaml)
    cfg_dir='configs',             # Directory containing config files
    strict=True                    # Enforce validation
)

# Access results
print(final_args)  # Returns dict with parsed arguments

Key Features:

• Automatically constructs file paths: {cfg_dir}/{name}.yaml
• Loads context from ctx.yaml if specified
• Parses defaults before applying config files
• Returns merged configuration as dictionary

---

Configuration Structure

Default Configuration (defaults.yaml)

arg1:
  default: ~                    # ~ means None
  help: "Description for arg1"
arg2:
  default: 42
  help: "Number of iterations"
arg3: True                      # Simple format: value directly
arg4: "string value"            # Simple format: string value

User Configuration (config1.yaml)

arg1: value1                    # Override default value
arg2: 100                       # Override another value

Context Management

Context YAML files support dynamic context management:

# ctx.yaml
import: !ipt
  torch.nn:
    relu: ReLU
    gelu: GELU

The !ipt tag can import multiple modules and attributes, which are then available for !ctx references.

---

Advanced: Custom YAML Tags

The library extends PyYAML with custom tags for dynamic value resolution.

!ctx - Context Variables

Reference variables defined in a context YAML file:

Context file (ctx.yaml):

import: !ipt
  torch.nn:
    relu: ReLU
    gelu: GELU

Usage in config:

activation: !ctx relu           # Resolves to "ReLU"
layer: !ctx gelu                # Resolves to "GELU"

!ipt - Import Attributes

Import attributes from Python modules:

import: !ipt
  datetime: [now]               # Resolves to datetime.datetime.now()
  os: [path, join]              # Resolves to os.path.join
  math: [sqrt, pi]              # Resolves to math.sqrt, math.pi

!call - Function Calls

Execute Python functions and use the result:

output_path: !call
  call: os.path.join
  part1: tmp
  part2: output

Resolves to: tmp/output

!join - Join Lists

Flatten and join list elements:

paths:
  - /usr/local
  - bin

Resolves to: ['/usr/local', 'bin']

!chain2path - Path Chain

Chain multiple path components:

output: !chain2path
  - tmp
  - output
  - data

Resolves to: tmp/output/data

---

Command-Line Options

Flag	Description	Example
-d, --cfg_dir	Directory containing YAML config files	-d configs
-c, --cfg	Names of configuration files to load	-c config1 config2
-dc, --default_yaml	Path to default YAML file	-dc defaults.yaml
-ctx, --ctx_yaml	Path to context YAML file	-ctx ctx.yaml
--strict	Enforce user config keys exist in defaults	--strict
--doc <arg>	Get help for specific argument(s)	--doc arg1 arg2
--docs	Print help for all arguments	--docs

Example Commands

# Load defaults and config files
python my_script.py -dc defaults.yaml -d configs -c config1 config2

# With context resolution
python my_script.py -dc defaults.yaml -ctx ctx.yaml -d configs -c config1

# Get help documentation
python my_script.py --docs
python my_script.py --doc arg1 arg2

# Strict mode (enforce config validation)
python my_script.py -dc defaults.yaml --strict -d configs -c config1

---

Troubleshooting

Best Practices

1. Start with defaults: Define comprehensive defaults in defaults.yaml
2. Use context files: Centralize reusable values in ctx.yaml
3. Document with help: Always provide help text for arguments
4. Validate with strict: Use --strict to catch configuration errors
5. Chain configs: Load multiple configs for modular configuration

Common Issues

Missing keys in strict mode

• Symptom: AssertionError: Argument 'X' not defined in the default configuration.
• Solution: Ensure all user config keys appear in defaults.yaml with proper structure (even if using minimal entries)

Context variables not resolving

• Symptom: !ctx xxx stays as literal string instead of resolving
• Check that ctx.yaml exists and is loaded via -ctx flag
• Verify import paths are correct in context yaml (import: !ipt)
• Variable names match between import and reference

Config merging order wrong

• Symptom: Config file values overwritten by defaults instead of overriding them
• Explanation: Priority order is: CLI args > user configs > defaults. Ensure config files load after defaults

---

License

MIT