yaml-test-params

A Python library for dynamic test parameter generation from YAML configuration files. This library enables flexible, data-driven test scenarios by combining Pydantic models with pytest's parametrize functionality or Python's built-in unittest library.

Features

• Configuration-driven tests: Define test parameters in YAML files instead of hardcoding them
• Pydantic validation: Type-safe configuration models with automatic validation
• Automatic test expansion: Range configurations are automatically expanded into individual test cases
• Seamless pytest integration: Works with pytest's native parametrize mechanism
• unittest support: Generated parameter sets can be iterated in standard unittest.TestCase tests
• Flexible parameter types: Support for simple values, lists, and ranges
• Custom YAML loading: Override the default yaml.safe_load behavior for tags, preprocessing, includes, or environment variables
• Custom test case data: test_cases can include any YAML data types accepted by your Pydantic models

Installation

uv add yaml-test-params

For pytest integration, install the pytest extra:

uv add "yaml-test-params[pytest]"

Dependencies:

• Python >= 3.9
• pydantic >= 2.0
• pyyaml >= 6.0.2
• pytest >= 8.4.2 (optional, required for pytest integration)

For local development and examples, install the development extra:

uv sync --extra dev

How It Works

The project supports dynamic parameter generation for tests from configuration files, enabling flexible test scenarios.

Workflow

1. Base Pydantic models define the structure of test cases and the YAML configuration file structure.
2. YAML configuration defines test parameters and scenarios.
3. Test runner integration uses the generated arguments either through the bundled pytest plugin or directly inside unittest.TestCase.

Quick Start

Step 1: Define Your Models

Create Pydantic models that represent your test case structure:

from yaml_test_params.models import (
    BaseTestCase,
    BaseTestConfig,
    BaseTestConfigCollection,
    ParametrizeInteger,
    ParametrizeString,
)


class ExampleTestCase(BaseTestCase):
    test_name: str
    integer: ParametrizeInteger
    string: ParametrizeString

    @property
    def arg_id(self) -> str:
        return self.test_name


class ExampleTestConfig(BaseTestConfig):
    test_cases: list[ExampleTestCase]


class ExampleTestConfigCollection(BaseTestConfigCollection):
    collection: list[ExampleTestConfig]

Step 2: Create YAML Configuration

Define your test parameters in a YAML file:

collection:
  - name: examples
    test_cases:
      - test_name: int_1,2,3__str_a
        integer:
          values: [1, 2, 3]
        string: a

      - test_name: int_42__str_a,b,c
        integer: 42
        string:
          values: [a, b, c]

      - test_name: int_1_10_1__str_a
        integer:
          from: 1
          to: 10
          step: 1
        string: a

      - test_name: int_1_10_2__str_a,b,c
        integer:
          from: 1
          to: 10
          step: 2
        string:
          values: [a, b, c]

Step 3: Use Generated Parameters in Tests

You can use the generated parameters with either pytest or unittest.

Option A: Use the pytest Plugin

Create a reusable configuration source and decorate only the methods that need YAML parametrization:

from yaml_test_params.pytest import YamlConfigSource, yaml_parametrize
from ..models import ExampleTestConfigCollection


EXAMPLE_CONFIGS = YamlConfigSource(
    path="examples/collection.yaml",
    model=ExampleTestConfigCollection,
)

The plugin is discovered automatically when both the package and pytest are installed. The pytest extra provides the required pytest dependency. Undecorated methods continue to run as ordinary pytest tests:

class TestParametrizeExamples:
    """Test class demonstrating pytest parametrize with integer and string variables."""

    @yaml_parametrize(EXAMPLE_CONFIGS, "examples")
    def test_values(self, test_name: str, integer: int, string: str):
        """Test that integer and string parameters are correctly passed."""
        print(f"\n==============\n")
        print(f"Test name: {test_name}")
        print(f"integer: {integer}\nstring: {string}")

    def test_value(self):
        """This method is not parametrized from YAML."""
        assert True

Run the pytest example:

uv run pytest -s examples/pytest_tests

If pytest plugin auto-loading is disabled, enable the plugin explicitly:

# conftest.py
pytest_plugins = ["yaml_test_params.pytest_plugin"]

For a project that already has its own pytest_generate_tests hook, call the public integration function instead:

from yaml_test_params.pytest import generate_yaml_tests


def pytest_generate_tests(metafunc):
    generate_yaml_tests(metafunc)

    # Additional project-specific parametrization can follow.

Automatic plugin loading, explicit pytest_plugins, and a manual generate_yaml_tests() call are alternative integration modes. Normally only one is needed; repeated processing of the same pytest metafunction is ignored.

Option B: Use unittest

Load the generated arguments once and iterate over them in a unittest.TestCase:

import unittest

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
)


class TestParametrizeExamples(unittest.TestCase):
    """Test class demonstrating unittest with generated YAML parameters."""

    cases = parametrize_args.argvalues

    def test_values(self):
        """Test that integer and string parameters are correctly passed."""
        for test_name, integer, string in self.cases:
            with self.subTest(test_name=test_name, integer=integer, string=string):
                print(f"\n==============\n")
                print(f"Test name: {test_name}")
                print(f"integer: {integer}\nstring: {string}")

Run the unittest example:

uv run python -m unittest examples.unittest_tests.test_examples

See the full examples in examples/pytest_tests and examples/unittest_tests.

Configuration Types

The library supports three types of parameter configurations:

test_cases may also contain any additional fields and data types that can be represented in YAML and validated by your Pydantic models, such as booleans, lists, dictionaries, nested models, dates, or enums. These fields are passed to generated test arguments according to the model definition.

The built-in parametrization config models expand int, str, and float values through ParametrizeInteger, ParametrizeString, and ParametrizeFloat.

class ExampleTestCase(BaseTestCase):
    integer: ParametrizeInteger
    string: ParametrizeString
    floating_point: ParametrizeFloat

test_cases:
  - test_name: int_1_10_2__str_a,b,c__float_0.1_0.3_0.1
    integer:
        from: 1
        to: 10
        step: 2
    string:
        values: [a, b, c]
    floating_point:
        from: 0.1
        to: 0.3
        step: 0.1

Simple Value

A single value for a parameter:

integer: 42
string: "hello"
floating_point: 0.25

List of Values

Multiple discrete values:

integer:
  values: [1, 2, 3]
string:
  values: [a, b, c]
floating_point:
  values: [0.1, 0.25, 0.5]

Range

A range of values with start, end, and step:

integer:
  from: 1
  to: 10
  step: 2

This generates values: [1, 3, 5, 7, 9].

step must be non-zero and point from from toward to: positive for an ascending range and negative for a descending range.

Descending ranges are also supported:

integer:
  from: 5
  to: 1
  step: -2

This generates values: [5, 3, 1].

Floating-point ranges use FloatRangeConfig:

floating_point:
  from: 0.1
  to: 0.3
  step: 0.1

This generates [0.1, 0.2, 0.3]. The range is calculated with Decimal arithmetic to avoid accumulating binary floating-point errors, then its values are passed to tests as float. As with integer ranges, step must be non-zero and its sign must match the range direction.

Available Models

ValueConfig

Configuration for parameters with a simple value:

class ValueConfig(BaseModel, Generic[T]):
    value: T

ListConfig

Configuration for parameters with a list of values:

class ListConfig(BaseModel, Generic[T]):
    values: list[T]

IntegerRangeConfig

Configuration for parameters with an integer range:

class IntegerRangeConfig(BaseModel):
    from_: int
    to: int
    step: int

RangeConfig remains available as a backwards-compatible alias for IntegerRangeConfig.

FloatRangeConfig

Configuration for a floating-point range calculated with Decimal:

class FloatRangeConfig(BaseModel):
    from_: Decimal
    to: Decimal
    step: Decimal

Type Aliases

ParametrizeIntegerConfigModels = IntegerRangeConfig | ListConfig[int] | ValueConfig[int]
ParametrizeStringConfigModels = ListConfig[str] | ValueConfig[str]
ParametrizeFloatConfigModels = FloatRangeConfig | ListConfig[float] | ValueConfig[float]
ParametrizeInteger = int | ParametrizeIntegerConfigModels
ParametrizeString = str | ParametrizeStringConfigModels
ParametrizeFloat = float | ParametrizeFloatConfigModels

Base Classes

class BaseTestCase(BaseModel, ABC):
    test_name: str

class BaseTestConfig(BaseModel):
    name: str
    test_cases: list[BaseTestCase]

class BaseTestConfigCollection(BaseModel):
    collection: list[BaseTestConfig]

API Reference

load_parametrize_args()

Loads and parses a YAML configuration file and returns parametrize arguments.

def load_parametrize_args(
    path_to_configs: Union[pathlib.Path, str],
    config_collection_model: Type[TestConfigCollection],
    collection_name: str,
    *,
    yaml_loader: YamlLoader = yaml.safe_load,
) -> ParametrizeArgs:

Parameters:

Parameter	Type	Description
path_to_configs	pathlib.Path | str	Path to the YAML configuration file
config_collection_model	Type[TestConfigCollection]	Pydantic model class for parsing the configuration
collection_name	str	Name of the test collection to use from the configuration
yaml_loader	Callable[[TextIO], Any]	Optional custom YAML loader. Defaults to yaml.safe_load

Returns: ParametrizeArgs object containing parametrize arguments

Raises: ValueError if no configuration is found for the given collection name

Custom YAML Loader

Use yaml_loader when you need custom YAML parsing, preprocessing, tags, includes, or environment variable substitution before Pydantic validation. The same loader can be used directly with load_parametrize_args() or through YamlConfigSource and the pytest plugin.

from typing import TextIO

import yaml


class CustomSafeLoader(yaml.SafeLoader):
    pass


def construct_times_two(loader, node):
    return int(loader.construct_scalar(node)) * 2


CustomSafeLoader.add_constructor("!times_two", construct_times_two)


def custom_yaml_loader(f: TextIO) -> dict:
    return yaml.load(f, Loader=CustomSafeLoader)

Use it when loading arguments directly:

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
    yaml_loader=custom_yaml_loader,
)

Or attach it to a reusable pytest configuration source:

from yaml_test_params.pytest import YamlConfigSource
from ..models import ExampleTestConfigCollection


EXAMPLE_CONFIGS = YamlConfigSource(
    path="examples/collection.yaml",
    model=ExampleTestConfigCollection,
    yaml_loader=custom_yaml_loader,
)

ParametrizeArgs

Dataclass holding generated test parameters for pytest and unittest integrations:

@dataclass
class ParametrizeArgs:
    argnames: str | None = None
    argvalues: list[tuple] = field(default_factory=list)
    ids: list[str] = field(default_factory=list)

Methods:

Method	Description
init_arg_names(model_cls)	Initialize argument names from a Pydantic model
add_params(arg_id, arg_values)	Add a parameterized test case
to_dict()	Convert to dictionary for metafunc.parametrize()
keys	Property returning the tuple of argument keys
keys_set	Property returning the set of argument keys

Exported Symbols

__all__ = [
    "BaseTestCase",
    "BaseTestConfig",
    "BaseTestConfigCollection",
    "FloatRangeConfig",
    "IntegerRangeConfig",
    "ListConfig",
    "ParametrizeArgs",
    "ParametrizeFloat",
    "ParametrizeFloatConfigModels",
    "ParametrizeInteger",
    "ParametrizeIntegerConfigModels",
    "ParametrizeString",
    "ParametrizeStringConfigModels",
    "RangeConfig",
    "ValueConfig",
    "load_parametrize_args",
]

Python Compatibility Tests

The project tests the latest compatible dependencies on Python 3.9 through 3.14. It also tests the minimum supported versions of Pydantic, PyYAML, and pytest on Python 3.9 through 3.11, where binary distributions for those versions are available.

Run the compatibility matrix through pytest:

uv run pytest python_compatibility_tests -v

Alternatively, run the standalone shell script:

./python_compatibility_tests/run_python_compatibility.sh

Both commands use isolated uv environments and leave the project's .venv unchanged. Missing Python versions are downloaded automatically by uv.

Project Structure

yaml-test-params/
├── examples/
│   ├── collection.yaml
│   ├── models.py
│   ├── pytest_tests/
│   │   ├── conftest.py
│   │   └── test_examples.py
│   └── unittest_tests/
│       └── test_examples.py
├── tests/
│   ├── test_args_loader.py
│   ├── test_models.py
│   ├── test_parametrize_args.py
│   └── test_pytest_integration.py
├── python_compatibility_tests/
│   ├── run_python_compatibility.sh
│   └── test_python_compatibility.py
├── yaml_test_params/
│   ├── __init__.py
│   ├── args_loader.py          # YAML configuration loader
│   ├── models.py               # Pydantic model definitions
│   ├── parametrize_args.py     # Parametrize arguments dataclass
│   ├── pytest.py               # Public pytest integration API
│   ├── pytest_plugin.py        # Automatically discovered pytest plugin
│   └── py.typed                # PEP 561 typing marker
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.txt
├── pyproject.toml
├── README.md
└── uv.lock

License

MIT