yaml-test-params

A Python library for dynamic test parameter generation from YAML configuration files. This library enables flexible, data-driven test scenarios by combining Pydantic models with pytest's parametrize functionality or Python's built-in unittest library.

Features

• Configuration-driven tests: Define test parameters in YAML files instead of hardcoding them
• Pydantic validation: Type-safe configuration models with automatic validation
• Automatic test expansion: Range configurations are automatically expanded into individual test cases
• Seamless pytest integration: Works with pytest's native parametrize mechanism
• unittest support: Generated parameter sets can be iterated in standard unittest.TestCase tests
• Flexible parameter types: Support for simple values, lists, and ranges
• Custom YAML loading: Override the default yaml.safe_load behavior for tags, preprocessing, includes, or environment variables
• Custom test case data: test_cases can include any YAML data types accepted by your Pydantic models

Installation

uv add yaml-test-params

Dependencies:

• Python >= 3.9
• pydantic >= 2.0
• pyyaml >= 6.0.2

For local development and examples, install the development extra:

uv sync --extra dev

How It Works

The project supports dynamic parameter generation for tests from configuration files, enabling flexible test scenarios.

Workflow

1. Base Pydantic models define the structure of test cases and the YAML configuration file structure.
2. YAML configuration defines test parameters and scenarios.
3. Test runner integration uses the generated arguments either through a pytest_generate_tests hook or directly inside unittest.TestCase.

Quick Start

Step 1: Define Your Models

Create Pydantic models that represent your test case structure:

from yaml_test_params.models import (
    BaseTestCase,
    BaseTestConfig,
    BaseTestConfigCollection,
    ParametrizeInteger,
    ParametrizeString,
)


class ExampleTestCase(BaseTestCase):
    test_name: str
    integer: ParametrizeInteger
    string: ParametrizeString

    @property
    def arg_id(self) -> str:
        return self.test_name


class ExampleTestConfig(BaseTestConfig):
    test_cases: list[ExampleTestCase]


class ExampleTestConfigCollection(BaseTestConfigCollection):
    collection: list[ExampleTestConfig]

Step 2: Create YAML Configuration

Define your test parameters in a YAML file:

collection:
  - name: examples
    test_cases:
      - test_name: int_1,2,3__str_a
        integer:
          values: [1, 2, 3]
        string: a

      - test_name: int_42__str_a,b,c
        integer: 42
        string:
          values: [a, b, c]

      - test_name: int_1_10_1__str_a
        integer:
          from: 1
          to: 10
          step: 1
        string: a

      - test_name: int_1_10_2__str_a,b,c
        integer:
          from: 1
          to: 10
          step: 2
        string:
          values: [a, b, c]

Step 3: Use Generated Parameters in Tests

You can use the generated parameters with either pytest or unittest.

Option A: Configure pytest Hook

Add the hook to your conftest.py:

import pytest

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


def pytest_generate_tests(metafunc: pytest.Metafunc) -> None:
    test_cls = getattr(metafunc, "cls", None)
    if test_cls is None:
        return

    full_cls_name = f"{test_cls.__module__}.{test_cls.__qualname__}"

    if (
        full_cls_name == "examples.pytest_tests.test_examples.TestParametrizeExamples"
        and metafunc.function.__name__ == "test_values"
    ):
        parametrize_args = load_parametrize_args(
            path_to_configs="examples/collection.yaml",
            config_collection_model=ExampleTestConfigCollection,
            collection_name="examples",
        )
        needed_args = parametrize_args.keys_set
        if needed_args.issubset(set(metafunc.fixturenames)):
            metafunc.parametrize(**parametrize_args.to_dict())

Then write tests that accept the parameters defined in your configuration:

class TestParametrizeExamples:
    """Test class demonstrating pytest parametrize with integer and string variables."""

    def test_values(self, test_name: str, integer: int, string: str):
        """Test that integer and string parameters are correctly passed."""
        print(f"\n==============\n")
        print(f"Test name: {test_name}")
        print(f"integer: {integer}\nstring: {string}")

Run the pytest example:

uv run pytest -s examples/pytest_tests

Option B: Use unittest

Load the generated arguments once and iterate over them in a unittest.TestCase:

import unittest

from yaml_test_params.args_loader import load_parametrize_args
from ..models import ExampleTestConfigCollection


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
)


class TestParametrizeExamples(unittest.TestCase):
    """Test class demonstrating unittest with generated YAML parameters."""

    cases = parametrize_args.argvalues

    def test_values(self):
        """Test that integer and string parameters are correctly passed."""
        for test_name, integer, string in self.cases:
            with self.subTest(test_name=test_name, integer=integer, string=string):
                print(f"\n==============\n")
                print(f"Test name: {test_name}")
                print(f"integer: {integer}\nstring: {string}")

Run the unittest example:

uv run python -m unittest examples.unittest_tests.test_examples

See the full examples in examples/pytest_tests and examples/unittest_tests.

Configuration Types

The library supports three types of parameter configurations:

test_cases may also contain any additional fields and data types that can be represented in YAML and validated by your Pydantic models, such as booleans, lists, dictionaries, nested models, dates, or enums. These fields are passed to generated test arguments according to the model definition.

The built-in parametrization config models currently expand only int and str values through ParametrizeInteger and ParametrizeString.

class ExampleTestCase(BaseTestCase):
    integer: ParametrizeInteger
    string: ParametrizeString
    list_of_types: list[int, str, float]

test_cases:
  - test_name: int_1_10_2__str_a,b,c__42_abc_0.07
    integer:
        from: 1
        to: 10
        step: 2
    string:
        values: [a, b, c]
    list_of_types: [42, abc, 0.07]

Simple Value

A single value for a parameter:

integer: 42
string: "hello"

List of Values

Multiple discrete values:

integer:
  values: [1, 2, 3]
string:
  values: [a, b, c]

Range

A range of values with start, end, and step:

integer:
  from: 1
  to: 10
  step: 2

This generates values: [1, 3, 5, 7, 9].

step must be a positive integer.

Available Models

ValueConfig

Configuration for parameters with a simple value:

class ValueConfig(BaseModel):
    value: int | str

ListConfig

Configuration for parameters with a list of values:

class ListConfig(BaseModel):
    values: list[int | str]

RangeConfig

Configuration for parameters with a range:

class RangeConfig(BaseModel):
    from_: int
    to: int
    step: int

Type Aliases

ParametrizeIntegerConfigModels = RangeConfig | ListConfig | ValueConfig
ParametrizeStringConfigModels = ListConfig | ValueConfig
ParametrizeInteger = int | ParametrizeIntegerConfigModels
ParametrizeString = str | ParametrizeStringConfigModels

Base Classes

class BaseTestCase(BaseModel, ABC):
    test_name: str

class BaseTestConfig(BaseModel):
    name: str
    test_cases: list[BaseTestCase]

class BaseTestConfigCollection(BaseModel):
    collection: list[BaseTestConfig]

API Reference

load_parametrize_args()

Loads and parses a YAML configuration file and returns parametrize arguments.

def load_parametrize_args(
    path_to_configs: Union[pathlib.Path, str],
    config_collection_model: Type[TestConfigCollection],
    collection_name: str,
    *,
    yaml_loader: YamlLoader = yaml.safe_load,
) -> ParametrizeArgs:

Parameters:

Parameter	Type	Description
path_to_configs	pathlib.Path | str	Path to the YAML configuration file
config_collection_model	Type[TestConfigCollection]	Pydantic model class for parsing the configuration
collection_name	str	Name of the test collection to use from the configuration
yaml_loader	Callable[[TextIO], Any]	Optional custom YAML loader. Defaults to yaml.safe_load

Returns: ParametrizeArgs object containing parametrize arguments

Raises: ValueError if no configuration is found for the given collection name

Custom YAML loader

Use yaml_loader when you need custom YAML parsing, preprocessing, tags, includes, or environment variable substitution before Pydantic validation.

from typing import TextIO

import yaml


def custom_yaml_loader(f: TextIO) -> dict:
    data = yaml.safe_load(f)
    # Modify or enrich the loaded YAML data here.
    return data


parametrize_args = load_parametrize_args(
    path_to_configs="examples/collection.yaml",
    config_collection_model=ExampleTestConfigCollection,
    collection_name="examples",
    yaml_loader=custom_yaml_loader,
)

ParametrizeArgs

Dataclass holding generated test parameters for pytest and unittest integrations:

@dataclass
class ParametrizeArgs:
    argnames: str | None = None
    argvalues: list[tuple] = field(default_factory=list)
    ids: list[str] = field(default_factory=list)

Methods:

Method	Description
init_arg_names(model_cls)	Initialize argument names from a Pydantic model
add_params(arg_id, arg_values)	Add a parameterized test case
to_dict()	Convert to dictionary for metafunc.parametrize()
keys	Property returning the tuple of argument keys
keys_set	Property returning the set of argument keys

Exported Symbols

__all__ = [
    "BaseTestCase",
    "BaseTestConfig",
    "BaseTestConfigCollection",
    "ListConfig",
    "ParametrizeArgs",
    "ParametrizeInteger",
    "ParametrizeIntegerConfigModels",
    "ParametrizeString",
    "ParametrizeStringConfigModels",
    "RangeConfig",
    "ValueConfig",
    "load_parametrize_args",
]

Project Structure

yaml-test-params/
├── examples/
│   ├── collection.yaml
│   ├── models.py
│   ├── pytest_tests/
│   │   ├── conftest.py
│   │   └── test_examples.py
│   └── unittest_tests/
│       └── test_examples.py
├── tests/
│   ├── test_args_loader.py
│   ├── test_models.py
│   └── test_parametrize_args.py
├── yaml_test_params/
│   ├── __init__.py
│   ├── args_loader.py          # YAML configuration loader
│   ├── models.py               # Pydantic model definitions
│   └── parametrize_args.py     # Parametrize arguments dataclass
├── CONTRIBUTING.md
├── LICENSE.txt
├── pyproject.toml
└── README.md

License

MIT