configz 3.0.1

Serialization stuff for config files

configz

Read the documentation!

configz is a YAML handling library that provides enhanced loading and dumping capabilities for YAML files. It builds upon PyYAML to offer additional features like environment variable support, file inclusion, and Jinja2 template resolution. Special mentions also to pyyaml_env_tag as well as pyyaml-include, this library exposes these YAML extensions with a unified interface.

Loading YAML

Basic Loading

To load YAML content from a string:

from configz import load_yaml

# Simple YAML loading
data = load_yaml("""
name: John
age: 30
""")

To load from a file:

from configz import load_yaml_file

# Load from local file
config = load_yaml_file("config.yaml")

# Load from remote file (S3, HTTP, etc.)
remote_config = load_yaml_file("s3://bucket/config.yaml")

Safety Modes

configz supports three safety modes when loading YAML:

# Safe mode - most restrictive, recommended for untrusted input
data = load_yaml(content, mode="safe")

# Full mode - allows some additional types but restricts dangerous ones
data = load_yaml(content, mode="full")

# Unsafe mode - allows all YAML features (default)
data = load_yaml(content, mode="unsafe")

Warning Always use "safe" mode when loading untrusted YAML content to prevent code execution vulnerabilities.

File Inclusion

configz supports including other YAML files using the !include tag:

# main.yaml
database:
  !include db_config.yaml
logging:
  !include logging_config.yaml

When loading, specify the base path for includes:

config = load_yaml_file("main.yaml", include_base_path="configs/")

Environment Variables

Use the !ENV tag to reference environment variables:

database:
  password: !ENV DB_PASSWORD
  host: !ENV ${DB_HOST:localhost}  # with default value

Template Resolution

configz can resolve Jinja2 templates in YAML:

from jinja2 import Environment
import configz

env = Environment()
yaml_content = """
message: "Hello {{ name }}!"
"""

data = load_yaml(
    yaml_content,
    resolve_strings=True,
    jinja_env=env
)

Inheritance

YAML files can inherit from other files using the INHERIT key. You can specify either a single file or a list of files to inherit from:

# Single inheritance
# prod.yaml
INHERIT: base.yaml
database:
  host: prod.example.com

# Multiple inheritance
# prod_with_logging.yaml
INHERIT:
  - base.yaml
  - logging.yaml
  - monitoring.yaml
database:
  host: prod.example.com

When using multiple inheritance, files are processed in order, with later files taking precedence over earlier ones. The current file's values take precedence over all inherited values.

For example:

# base.yaml
database:
  host: localhost
  port: 5432
  timeout: 30

# logging.yaml
database:
  timeout: 60
logging:
  level: INFO

# prod.yaml
INHERIT:
  - base.yaml
  - logging.yaml
database:
  host: prod.example.com

When loading prod.yaml, the final configuration will be:

database:
  host: prod.example.com  # from prod.yaml
  port: 5432             # from base.yaml
  timeout: 60            # from logging.yaml
logging:
  level: INFO            # from logging.yaml

Load with inheritance enabled:

config = load_yaml_file("prod.yaml", resolve_inherit=True)

Dumping YAML

To serialize Python objects to YAML:

from configz import dump_yaml

data = {
    "name": "John",
    "scores": [1, 2, 3],
    "active": True
}

yaml_string = dump_yaml(data)

Dataclasses and Pydantic v2 models can also get dumped.

Custom Class Mapping

Map custom classes to built-in types for YAML representation:

from collections import OrderedDict

data = OrderedDict([("b", 2), ("a", 1)])
yaml_string = dump_yaml(data, class_mappings={OrderedDict: dict})

Custom Loader Configuration

For advanced use cases, you can create a custom loader:

from configz import get_loader
import yaml

# Create custom loader with specific features
loader_cls = get_loader(
    yaml.SafeLoader,
    include_base_path="configs/",
    enable_include=True,
    enable_env=True,
    resolve_strings=True,
    jinja_env=jinja_env,
    type_converters={int: str}
)

# Use custom loader
data = yaml.load(content, Loader=loader_cls)

Custom Tag Handling

configz provides a YAMLParser class for handling custom YAML tags. This allows you to define how specific tagged values should be processed during YAML loading.

Basic Tag Registration

You can register tag handlers using either a decorator or explicit registration:

from configz import YAMLParser

# Create parser instance
yaml_parser = YAMLParser()

# register handler
def handle_uppercase(data: str) -> str:
    return data.upper()

yaml_parser.register_handler("uppercase", handle_uppercase)

Using Custom Tags

Once registered, you can use the custom tags in your YAML:

# config.yaml
message: !uppercase "hello world"

Load the YAML using the parser:

# Load from string
data = yaml_parser.load_yaml("""
message: !uppercase "hello world"
""")

# Or load from file
data = yaml_parser.load_yaml_file("config.yaml")

print(data["message"])  # "HELLO WORLD"

Class Registration

For simple cases where you just want to convert tagged dictionary data into class instances, you can use the register_class method. The method will automatically use the lowercase class name as the tag name (following YAML tag conventions), but you can override this with a custom tag name:

from configz import YAMLParser
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int

@dataclass
class HomeAddress:
    street: str
    city: str

yaml_parser = YAMLParser()

# Register using class name as tag (will use "person" as tag)
yaml_parser.register_class(Person)

# Register with custom tag name instead of "homeaddress"
yaml_parser.register_class(HomeAddress, "address")

# Now you can use it in YAML:
data = yaml_parser.load_yaml("""
user: !person
  name: John Doe
  age: 30
home: !address
  street: 123 Main St
  city: New York
""")

print(data["user"])    # Person(name='John Doe', age=30)
print(data["home"])    # HomeAddress(street='123 Main St', city='New York')

Complex Structures

Custom tags can be used in nested structures and lists:

team:
  manager: !person
    name: Alice Smith
    age: 45
  members:
    - !person
      name: Bob Johnson
      age: 30
    - !person
      name: Carol White
      age: 28
messages:
  - !uppercase "welcome"
  - !uppercase "goodbye"

Combining with Other Features

The YAMLParser class supports all of configz's standard features:

data = yaml_parser.load_yaml_file(
    "config.yaml",
    mode="safe",                    # Safety mode
    include_base_path="configs/",   # For !include directives
    resolve_strings=True,           # Enable Jinja2 template resolution
    resolve_inherit=True,           # Enable inheritance
    jinja_env=jinja_env           # Custom Jinja2 environment
)

Available Tags

You can list all registered tags:

tags = yaml_parser.list_tags()
print(tags)  # ['!person', '!uppercase']

Universal load / dump interface

configz provides a universal load function that can handle YAML, JSON, TOML, and INI files. Apart from yaml, only stdlib modules are used, so no additional dependencies are required. Here's a simple example:

import configz

# Load files based on their extension
config = configz.load_file("config.yaml")    # YAML
settings = configz.load_file("settings.json") # JSON
params = configz.load_file("params.toml")    # TOML

# Or explicitly specify the format
data = configz.load_file("config.txt", mode="yaml")

# Load directly from strings
yaml_text = """
name: John
age: 30
"""
data = configz.load(yaml_text, mode="yaml")
# same in other direction
json_text = configz.dump(data, mode="json")
yaml.dump_file("config.yaml", data)

JSON Tools

Cross-platform JSON handling that works in all environments:

from configz import load_json, dump_json, JsonLoadError, JsonDumpError

# Load JSON from various sources
data = load_json('{"key": "value"}')  # From string
data = load_json(Path("config.json"))  # From file
data = load_json(b'{"key": "value"}')  # From bytes

# Dump JSON to various targets
json_str = dump_json(data)  # To string
dump_json(data, Path("output.json"))  # To file
json_bytes = dump_json(data, return_bytes=True)  # To bytes

# Error handling
try:
    data = load_json('invalid json')
except JsonLoadError as e:
    print(f"Failed to parse JSON: {e}")

try:
    dump_json(set())  # Sets aren't JSON serializable
except JsonDumpError as e:
    print(f"Failed to serialize: {e}")

Note If orjson is installed, the loader will automatically use it for JSON parsing / dumping, offering significantly better performance compared to the standard json module.