Cache assertion data to simplify regression testing of complex serializable data
Project description
pytest_cache_assert
Cache assertion data to simplify regression testing of complex serializable data
Installation
poetry add pytest_assert_check --dev
Quick Start
The primary use case of this package is regression testing of large dictionaries. You may have some parameterized test cases where you need to assert that a resulting dictionary is the same, but you don’t want to manually generate the expected fields and values and couple the test case to the source code. Instead you can cache or “record” the expected serialized data structure, check the expected dictionary into version control, then regenerate on changes
This package can minimize test case logic, while improving test thoroughness
This project was heavily inspired by the excellent pytest-recording
Basic Example
Situation: You've created a new project with poetry called package_a
with one file source_file.py
and test tests/test_file.py
"""package_a/source_file.py"""
import sys
from datetime import datetime
from typing import Any, Dict, List, Optional
from beartype import beartype
from pydantic import BaseModel
class User(BaseModel): # noqa: H601
"""Example from pydantic documentation."""
id: int # noqa: A003,VNE003
name = 'John Doe'
signup_ts: Optional[datetime] = None
friends: List[int] = []
@beartype
def create_data(name: str) -> Dict[str, Any]:
"""Arbitrary function that returns a dictionary.
This demonstration uses pydantic, but any dictionary can be tested!
"""
return User(id=sys.maxsize, name=name).dict()
"""tests/test_file.py"""
import pytest
@pytest.mark.parametrize('name', ['Test Name 1', 'Test Name 2'])
def test_create_data(name, assert_against_cache):
"""Basic test of create_data()."""
result = create_data(name=name)
# One could manually create the expected dictionary
cache = {'id': 9223372036854775807, 'signup_ts': None, 'friends': [], 'name': name}
assert result == cache
# ----------------------------------------------------------------------------------
# Or utilize the pytest_cache_assert fixture to compare against the last cached version
assert_against_cache(result)
pytest_cache_assert
will automatically create: tests/cache-assert/source_file/test_file/test_create_data-[Test Name 1].json
(and test_create_data[Test Name 2].json
) for each of the parameters when first run by caching the result
. Below is the example for test_create_data-[Test Name 1].json
. The cached files should be checked into version control. They can be manually edited if needed or regenerated by deleting the file and re-running the tests suite.
{
"_info": {
"func_args": {
"name": "Test Name 1"
},
"test_file": ".../tests/test_file.py",
"test_name": "test_create_data"
},
"_json": {
"friends": [],
"id": 9223372036854775807,
"name": "Test Name 1",
"signup_ts": null
}
}
More Examples
In your cached dictionary, you may have variable values with more complex logic to verify, such as dates, UUIDs, etc. These can be selectively ignored, matched-if-null, or some other user-specified check:
from uuid import uuid4
from datetime import datetime
from pytest_cache_assert import assert_against_cache
# TODO: Implement this example! Add as red test, then implement!
def test_variable_cache(assert_against_cache):
"""Demonstration of specifying custom assertion rules."""
# TODO: support list indexing on top of jmespath...
result = {"date": datetime.now(), {"nested": {"uuid": uuid4()}}}
assert_against_cache(result, ...)
Or you may want to write your own custom checks against the serialized data, such as with Cerberus or another library. This is possible with the validator
callable. The default is a no-op and that may be replaced with a custom function that raises an Exception on error.
# TODO: Add example for custom cerberus checks with `validator()`
Even More Example
For more examples, see Scripts or Tests
Global Configuration Options
custom directory
(set in pytest fixture like pytest-record)- Default is test directory/cache-assert/
record_rule
: regenerate on failure (re-raises assert, but updates cache). Default is to record once
import pytest
# TODO: Not currently used...
@pytest.fixture(scope='module')
def cache_assert_config():
return {
'rel_path_cache_dir': '..........',
'record_mode': 'rewrite', # TODO: Consider record mode
'filter_headers': ['authorization'], # TODO: Consider filters
}
Roadmap
See the Open Issues
and Milestones
for current status and ./docs/CODE_TAG_SUMMARY.md for annotations in the source code.
For release history, see the ./docs/CHANGELOG.md
Contributing
See the Developer Guide, Contribution Guidelines, etc
- ./docs/DEVELOPER_GUIDE.md
- ./docs/STYLE_GUIDE.md
- ./docs/CONTRIBUTING.md
- ./docs/CODE_OF_CONDUCT.md
- ./docs/SECURITY.md
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_cache_assert-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8cd948accbff441d907652dff9ac153ab5161bf0fe5e8fecf558c87208e5c294 |
|
MD5 | 05a2006a845ed6eb2fe1c5e646e661df |
|
BLAKE2b-256 | 5dc21d288c29bfa5688e3e4fb02135e192353f5f07eb078a7d5b6310ebd3b381 |
Hashes for pytest_cache_assert-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a0ce63d756a41264a9e3ca13939d36bb2801d854d7c001a3420d318cb96441e7 |
|
MD5 | ec3afcb960e5c772547f276d838244c5 |
|
BLAKE2b-256 | f7ea84239f2e2f57a0f6ad13a4289db858ebef375105234888d5fc5ccef8ebb4 |