pytest-accept 0.2.4

No project description provided

pytest-accept

pytest-accept is a pytest plugin for automatically updating outputs. It runs along with pytest, observes the generated outputs, and writes them to the test's documented outputs.

Before

def extract_functions(code):
    return re.findall(r"(\w+)\(", code)


def test_extract_functions():
    assert extract_functions("print('hi')") == ["print"]
    assert extract_functions("sum(map(f, x))") == ["sum"]

After pytest --accept

def test_extract_functions():
    assert extract_functions("print('hi')") == ["print"]
-   assert extract_functions("sum(map(f, x))") == ["sum"]
+   assert extract_functions("sum(map(f, x))") == ["sum", "map"]

pytest-accept is decoupled from the tests it works with — it can be used with existing tests, and the tests it edits are no different from normal tests. It works with both doctests and normal assert statements.

Installation

uv tool install -U pytest-accept

Or with pip:

pip install pytest-accept

To run, just pass --accept to pytest:

pytest --accept

Why?

• Often it's fairly easy to observe whether something is working by viewing the output it produces
• ...but often the output is verbose, and copying and pasting the output into the test is tedious
• pytest-accept does the copying & pasting for you
• Similarly, lots of folks generally find writing any tests a bit annoying, and prefer to develop by "running the code and seeing if it works". This library aims to make testing a joyful part of that development loop

This style of testing is fairly well-developed in some languages, although still doesn't receive the attention I think it deserves, and historically hasn't had good support in python.

The best explanation I've seen on this testing style is from @yminsky in a Jane Street Blogpost. @matklad also has an excellent summary in his blog post How to Test.

How it works

pytest-accept:

• Intercepts test failures from both doctests and assert statements
• Parses the files to understand where the documented values are
• Updates the documented values to match the generated values
• Writes everything back atomically

Things to know:

• Simple comparisons only: Assert rewriting only works with == comparisons against literals or simple expressions
• Overwrite by default: Pass --accept-copy to write to .py.new files instead.

Doctest quirks

Doctests are great for examples, but they have quirks

• Use raw strings for examples with backslashes:

  r"""
  >>> print("\n")
  \n
  """
  

• We handle blank lines automatically:

  """
  >>> print("one\n\ntwo")
  one
  <BLANKLINE>
  two
  """
  

• Really long outputs get truncated so they won't break your editor

Prior art

This testing style goes by many names: "snapshot testing", "regression testing", "expect testing", "literate testing", or "acceptance testing". Whatever the name, the pattern is the same: write tests, see what they produce, accept what's correct.

@matklad has an excellent overview in How to Test. The approach is well-established in many languages:

• cram (command-line tests)
• ppx_expect (OCaml)
• insta (Rust)

For more complex test scenarios, consider:

• pytest-regtest for file-based testing
• syrupy for snapshot testing
• pytest-insta for insta-style review

thanks to @untiaker, who found how to expand the original doctest solution into an approach that works with standard assert statements.