yaml-test-params

A Python library for dynamic test parameter generation from YAML configuration files. This library enables flexible, data-driven test scenarios by combining Pydantic models with pytest's parametrize functionality or Python's built-in unittest library.

Features

• Configuration-driven tests: Define test parameters in YAML files instead of hardcoding them
• Pydantic validation: Type-safe configuration models with automatic validation
• Automatic test expansion: Range configurations are automatically expanded into individual test cases
• Seamless pytest integration: Works with pytest's native parametrize mechanism
• unittest support: Generated parameter sets can be iterated in standard unittest.TestCase tests
• Flexible parameter types: Support for simple values, lists, and ranges
• Custom YAML loading: Override the default yaml.safe_load behavior for tags, preprocessing, includes, or environment variables
• Custom test case data: test_cases can include any YAML data types accepted by your Pydantic models

Installation

uv add yaml-test-params

Dependencies:

• Python >= 3.9
• pydantic >= 2.0
• pyyaml >= 6.0.2

For local development and examples, install the development extra:

uv sync --extra dev

How It Works

The project supports dynamic parameter generation for tests from configuration files, enabling flexible test scenarios.

Workflow

1. Base Pydantic models define the structure of test cases and the YAML configuration file structure.
2. YAML configuration defines test parameters and scenarios.
3. Test runner integration uses the generated arguments either through a pytest_generate_tests hook or directly inside unittest.TestCase.

Quick Start

Step 1: Define Your Models

Create Pydantic models that represent your test case structure:

from yaml_test_params.models import (
    BaseTestCase,
    BaseTestConfig,
    BaseTestConfigCollection,
    ParametrizeInteger,
    ParametrizeString,
)


class ExampleTestCase(BaseTestCase):
    test_name: str
    integer: ParametrizeInteger
    string: ParametrizeString

    @property
    def arg_id(self) -> str:
        return self.test_name


class ExampleTestConfig(BaseTestConfig):
    test_cases: list[ExampleTestCase]


class ExampleTestConfigCollection(BaseTestConfigCollection):
    collection: list[ExampleTestConfig]

Step 2: Create YAML Configuration

Define your test parameters in a YAML file:

collection:
  - name: examples
    test_cases:
      - test_name: int_1,2,3__str_a
        integer:
          values: [1, 2, 3]
        string: a

      - test_name: int_42__str_a,b,c
        integer: 42
        string:
          values: [a, b, c]

      - test_name: int_1_10_1__str_a
        integer:
          from: 1
          to: 10
          step: 1
        string: a

      - test_name: int_1_10_2__str_a,b,c
        integer:
          from: 1
          to: 10
          step: 2
        string:
          values: [a, b, c]

Step 3: Use Generated Parameters in Tests

You can use the generated parameters with either pytest or unittest.

Option A: Configure pytest Hook

Add the hook to your conftest.py:

import pytest

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


def pytest_generate_tests(metafunc: pytest.Metafunc) -> None:
    test_cls = getattr(metafunc, "cls", None)
    if test_cls is None:
        return

    full_cls_name = f"{test_cls.__module__}.{test_cls.__qualname__}"

    if (
        full_cls_name == "examples.pytest_tests.test_examples.TestParametrizeExamples"
        and metafunc.function.__name__ == "test_values"
    ):
        parametrize_args = load_parametrize_args(
            path_to_configs="examples/collection.yaml",
            config_collection_model=ExampleTestConfigCollection,
            collection_name="examples",
        )
        needed_args = parametrize_args.keys_set
        if needed_args.issubset(set(metafunc.fixturenames)):
            metafunc.parametrize(**parametrize_args.to_dict())

Then write tests that accept the parameters defined in your configuration:

class TestParametrizeExamples:
    """Test class demonstrating pytest parametrize with integer and string variables."""

    def test_values(self, test_name: str, integer: int, string: str):
        """Test that integer and string parameters are correctly passed."""
        print(f"\n==============\n")
        print(f"Test name: {test_name}")
        print(f"integer: {integer}\nstring: {string}")

Run the pytest example:

uv run pytest -s examples/pytest_tests

Option B: Use unittest

Load the generated arguments once and iterate over them in a unittest.TestCase:

import unittest

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
)


class TestParametrizeExamples(unittest.TestCase):
    """Test class demonstrating unittest with generated YAML parameters."""

    cases = parametrize_args.argvalues

    def test_values(self):
        """Test that integer and string parameters are correctly passed."""
        for test_name, integer, string in self.cases:
            with self.subTest(test_name=test_name, integer=integer, string=string):
                print(f"\n==============\n")
                print(f"Test name: {test_name}")
                print(f"integer: {integer}\nstring: {string}")

Run the unittest example:

uv run python -m unittest examples.unittest_tests.test_examples

See the full examples in examples/pytest_tests and examples/unittest_tests.

Configuration Types

The library supports three types of parameter configurations:

test_cases may also contain any additional fields and data types that can be represented in YAML and validated by your Pydantic models, such as booleans, lists, dictionaries, nested models, dates, or enums. These fields are passed to generated test arguments according to the model definition.

The built-in parametrization config models expand int, str, and float values through ParametrizeInteger, ParametrizeString, and ParametrizeFloat.

class ExampleTestCase(BaseTestCase):
    integer: ParametrizeInteger
    string: ParametrizeString
    floating_point: ParametrizeFloat

test_cases:
  - test_name: int_1_10_2__str_a,b,c__float_0.1_0.3_0.1
    integer:
        from: 1
        to: 10
        step: 2
    string:
        values: [a, b, c]
    floating_point:
        from: 0.1
        to: 0.3
        step: 0.1

Simple Value

A single value for a parameter:

integer: 42
string: "hello"
floating_point: 0.25

List of Values

Multiple discrete values:

integer:
  values: [1, 2, 3]
string:
  values: [a, b, c]
floating_point:
  values: [0.1, 0.25, 0.5]

Range

A range of values with start, end, and step:

integer:
  from: 1
  to: 10
  step: 2

This generates values: [1, 3, 5, 7, 9].

step must be non-zero and point from from toward to: positive for an ascending range and negative for a descending range.

Descending ranges are also supported:

integer:
  from: 5
  to: 1
  step: -2

This generates values: [5, 3, 1].

Floating-point ranges use FloatRangeConfig:

floating_point:
  from: 0.1
  to: 0.3
  step: 0.1

This generates [0.1, 0.2, 0.3]. The range is calculated with Decimal arithmetic to avoid accumulating binary floating-point errors, then its values are passed to tests as float. As with integer ranges, step must be non-zero and its sign must match the range direction.

Available Models

ValueConfig

Configuration for parameters with a simple value:

class ValueConfig(BaseModel, Generic[T]):
    value: T

ListConfig

Configuration for parameters with a list of values:

class ListConfig(BaseModel, Generic[T]):
    values: list[T]

IntegerRangeConfig

Configuration for parameters with an integer range:

class IntegerRangeConfig(BaseModel):
    from_: int
    to: int
    step: int

RangeConfig remains available as a backwards-compatible alias for IntegerRangeConfig.

FloatRangeConfig

Configuration for a floating-point range calculated with Decimal:

class FloatRangeConfig(BaseModel):
    from_: Decimal
    to: Decimal
    step: Decimal

Type Aliases

ParametrizeIntegerConfigModels = IntegerRangeConfig | ListConfig[int] | ValueConfig[int]
ParametrizeStringConfigModels = ListConfig[str] | ValueConfig[str]
ParametrizeFloatConfigModels = FloatRangeConfig | ListConfig[float] | ValueConfig[float]
ParametrizeInteger = int | ParametrizeIntegerConfigModels
ParametrizeString = str | ParametrizeStringConfigModels
ParametrizeFloat = float | ParametrizeFloatConfigModels

Base Classes

class BaseTestCase(BaseModel, ABC):
    test_name: str

class BaseTestConfig(BaseModel):
    name: str
    test_cases: list[BaseTestCase]

class BaseTestConfigCollection(BaseModel):
    collection: list[BaseTestConfig]

API Reference

load_parametrize_args()

Loads and parses a YAML configuration file and returns parametrize arguments.

def load_parametrize_args(
    path_to_configs: Union[pathlib.Path, str],
    config_collection_model: Type[TestConfigCollection],
    collection_name: str,
    *,
    yaml_loader: YamlLoader = yaml.safe_load,
) -> ParametrizeArgs:

Parameters:

Parameter	Type	Description
path_to_configs	pathlib.Path | str	Path to the YAML configuration file
config_collection_model	Type[TestConfigCollection]	Pydantic model class for parsing the configuration
collection_name	str	Name of the test collection to use from the configuration
yaml_loader	Callable[[TextIO], Any]	Optional custom YAML loader. Defaults to yaml.safe_load

Returns: ParametrizeArgs object containing parametrize arguments

Raises: ValueError if no configuration is found for the given collection name

Custom YAML loader

Use yaml_loader when you need custom YAML parsing, preprocessing, tags, includes, or environment variable substitution before Pydantic validation.

from typing import TextIO

import yaml


def custom_yaml_loader(f: TextIO) -> dict:
    data = yaml.safe_load(f)
    # Modify or enrich the loaded YAML data here.
    return data


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
    yaml_loader=custom_yaml_loader,
)

ParametrizeArgs

Dataclass holding generated test parameters for pytest and unittest integrations:

@dataclass
class ParametrizeArgs:
    argnames: str | None = None
    argvalues: list[tuple] = field(default_factory=list)
    ids: list[str] = field(default_factory=list)

Methods:

Method	Description
init_arg_names(model_cls)	Initialize argument names from a Pydantic model
add_params(arg_id, arg_values)	Add a parameterized test case
to_dict()	Convert to dictionary for metafunc.parametrize()
keys	Property returning the tuple of argument keys
keys_set	Property returning the set of argument keys

Exported Symbols

__all__ = [
    "BaseTestCase",
    "BaseTestConfig",
    "BaseTestConfigCollection",
    "FloatRangeConfig",
    "IntegerRangeConfig",
    "ListConfig",
    "ParametrizeArgs",
    "ParametrizeFloat",
    "ParametrizeFloatConfigModels",
    "ParametrizeInteger",
    "ParametrizeIntegerConfigModels",
    "ParametrizeString",
    "ParametrizeStringConfigModels",
    "RangeConfig",
    "ValueConfig",
    "load_parametrize_args",
]

Python Compatibility Tests

The project tests the latest compatible dependencies on Python 3.9 through 3.14. It also tests the minimum supported versions of Pydantic, PyYAML, and pytest on Python 3.9 through 3.11, where binary distributions for those versions are available.

Run the compatibility matrix through pytest:

uv run pytest python_compatibility_tests -v

Alternatively, run the standalone shell script:

./python_compatibility_tests/run_python_compatibility.sh

Both commands use isolated uv environments and leave the project's .venv unchanged. Missing Python versions are downloaded automatically by uv.

Project Structure

yaml-test-params/
├── examples/
│   ├── collection.yaml
│   ├── models.py
│   ├── pytest_tests/
│   │   ├── conftest.py
│   │   └── test_examples.py
│   └── unittest_tests/
│       └── test_examples.py
├── tests/
│   ├── test_args_loader.py
│   ├── test_models.py
│   └── test_parametrize_args.py
├── python_compatibility_tests/
│   ├── run_python_compatibility.sh
│   └── test_python_compatibility.py
├── yaml_test_params/
│   ├── __init__.py
│   ├── args_loader.py          # YAML configuration loader
│   ├── models.py               # Pydantic model definitions
│   └── parametrize_args.py     # Parametrize arguments dataclass
├── CONTRIBUTING.md
├── LICENSE.txt
├── pyproject.toml
└── README.md

License

MIT