readme-example-tester

Do not let your README/documentation examples become invalid: unit test them!

Install

pip install readme-example-tester

Usage

Mark the code blocks that you want to test.

Check out this crazy snippet:

<!-- example-id: tests/sample_pump_it_up.py -->
```python
def pump_it_up(input):
    return input + 100
```

Then create a unit test for that README, such as:

from pathlib import Path

from readme_example_tester import ReadmeTestCase


class TestReadme(ReadmeTestCase):
    # assuming test is in tests/ and README.md in root
    README_PATH = Path(__file__).parent.parent / 'README.md'
    TESTS_DIR = Path(__file__).parent

Whenever this test case runs, 3 tests are executed:

• test_readme_example_targets_have_tests: ensures every snippet is covered by a test case
• test_readme_code_blocks_match_example_targets: ensures every snippet matches an existing file
• test_examples_are_still_in_use: ensures all files inside TESTS_DIR matching SAMPLE_FILE_GLOB (sample_*) are still being used in the README

These tests, plus the sample-file coverage checks, keep README examples honest.

Dogfooding

This project README is actually covered by tests/test_dogfood.py

And the inner example is in tests/sample_one.md and covered by tests/test_sample_one_readme.py

Advanced usage

Custom marker

class DemoReadme(ReadmeTestCase):
    README_PATH = Path(__file__).parent / 'README.md'
    TESTS_DIR = Path(__file__).parent
    README_MARKER = 'demo-id'

<!-- demo-id: tests/some_sample.py -->
```python
def pump_it_up(input):
    return input + 100
```

Assert snippet outputs

Use <!-- example-id-output: ... --> (or README_MARKER-output if customized) to compare/validate outputs from an executable sample script.

Expanding the previous example to be executable

#!/usr/bin/env python3


def pump_it_up(input):
    return input + 100


def main(argv):
    if not argv:
        print('No pump')
    else:
        print(pump_it_up(int(argv[0])))


if __name__ == '__main__':
    import sys

    main(sys.argv[1:])

<!-- example-id-output: tests/sample_pump_it_up_cli.py 2 -->
```text
$ tests/sample_pump_it_up_cli.py 2
102
```

Partial snippets

Use # README+++ / # README--- in a sample file to narrow the excerpt that matches the README block when the source file has extra boilerplate.

If one sample file feeds multiple README blocks, use # README:<id>+++ / # README:<id>--- to split the source into named sections and then use <!-- example-id:<id> tests/... --> in the code snippets

Non-sample sample files

If you have a test file that matches SAMPLE_FILE_GLOB but it is not expected to be used in README, you can add # README-EXCLUDE

Development

Project setup and local checks live in CONTRIBUTING.md.