pytest-cache-assert 0.1.0

Cache assertion data to simplify regression testing of complex serializable data

pytest_cache_assert

Cache assertion data to simplify regression testing of complex serializable data

Installation

poetry add pytest_assert_check --dev

Quick Start

The primary use case of this package is regression testing of large dictionaries. You may have some parameterized test cases where you need to assert that a resulting dictionary is the same, but you don’t want to manually generate the expected fields and values and couple the test case to the source code. Instead you can cache or “record” the expected serialized data structure, check the expected dictionary into version control, then regenerate on changes

This package can minimize test case logic, while improving test thoroughness

This project was heavily inspired by the excellent pytest-recording

Basic Example

Situation: You've created a new project with poetry called package_a with one file source_file.py and test tests/test_file.py

"""package_a/source_file.py"""

import sys
from datetime import datetime
from typing import Any, Dict, List, Optional

from beartype import beartype
from pydantic import BaseModel


class User(BaseModel):  # noqa: H601
    """Example from pydantic documentation."""

    id: int  # noqa: A003,VNE003
    name = 'John Doe'
    signup_ts: Optional[datetime] = None
    friends: List[int] = []


@beartype
def create_data(name: str) -> Dict[str, Any]:
    """Arbitrary function that returns a dictionary.

    This demonstration uses pydantic, but any dictionary can be tested!

    """
    return User(id=sys.maxsize, name=name).dict()

"""tests/test_file.py"""

import pytest


@pytest.mark.parametrize('name', ['Test Name 1', 'Test Name 2'])
def test_create_data(name, assert_against_cache):
    """Basic test of create_data()."""
    result = create_data(name=name)

    # One could manually create the expected dictionary
    cache = {'id': 9223372036854775807, 'signup_ts': None, 'friends': [], 'name': name}
    assert result == cache
    # ----------------------------------------------------------------------------------
    # Or utilize the pytest_cache_assert fixture to compare against the last cached version
    assert_against_cache(result)

pytest_cache_assert will automatically create: tests/cache-assert/source_file/test_file/test_create_data-[Test Name 1].json (and test_create_data[Test Name 2].json) for each of the parameters when first run by caching the result. Below is the example for test_create_data-[Test Name 1].json. The cached files should be checked into version control. They can be manually edited if needed or regenerated by deleting the file and re-running the tests suite.

{
  "_info": {
    "func_args": {
      "name": "Test Name 1"
    },
    "test_file": ".../tests/test_file.py",
    "test_name": "test_create_data"
  },
  "_json": {
    "friends": [],
    "id": 9223372036854775807,
    "name": "Test Name 1",
    "signup_ts": null
  }
}

More Examples

In your cached dictionary, you may have variable values with more complex logic to verify, such as dates, UUIDs, etc. These can be selectively ignored, matched-if-null, or some other user-specified check:

from uuid import uuid4
from datetime import datetime

from pytest_cache_assert import assert_against_cache

# TODO: Implement this example! Add as red test, then implement!

def test_variable_cache(assert_against_cache):
    """Demonstration of specifying custom assertion rules."""
    # TODO: support list indexing on top of jmespath...
    result = {"date": datetime.now(), {"nested": {"uuid": uuid4()}}}

    assert_against_cache(result, ...)

Or you may want to write your own custom checks against the serialized data, such as with Cerberus or another library. This is possible with the validator callable. The default is a no-op and that may be replaced with a custom function that raises an Exception on error.

# TODO: Add example for custom cerberus checks with `validator()`

Even More Example

For more examples, see Scripts or Tests

Global Configuration Options

• custom directory (set in pytest fixture like pytest-record)
  • Default is test directory/cache-assert/
• record_rule: regenerate on failure (re-raises assert, but updates cache). Default is to record once

import pytest


# TODO: Not currently used...
@pytest.fixture(scope='module')
def cache_assert_config():
    return {
        'rel_path_cache_dir': '..........',
        'record_mode': 'rewrite',  # TODO: Consider record mode
        'filter_headers': ['authorization'],  # TODO: Consider filters
    }

Roadmap

See the Open Issues and Milestones for current status and ./docs/CODE_TAG_SUMMARY.md for annotations in the source code.

For release history, see the ./docs/CHANGELOG.md

Contributing

See the Developer Guide, Contribution Guidelines, etc

• ./docs/DEVELOPER_GUIDE.md
• ./docs/STYLE_GUIDE.md
• ./docs/CONTRIBUTING.md
• ./docs/CODE_OF_CONDUCT.md
• ./docs/SECURITY.md

License

LICENSE