Skip to main content

docattest for Python

docattest for Python projects: documentation that CI can fail. Your Markdown pages carry examples that run, facts that are compared with values in your code, tables whose rows are tests, and citations of tests you already have.

This package is what a Python user installs. It provides two commands:

  • docattest, the command you run. It is a launcher: it finds the docattest core (one binary for every language, which reads the pages and writes the report), tells it how to start the Python adapter and what command runs docattest in your project, and becomes it.
  • docattest-adapter-python, the adapter: the program that answers everything that needs Python. It speaks version 1 of the adapter protocol. You do not run it yourself.

The distribution is called docattest . Its import packages are docattest (the launcher) and docattest_python (the adapter).

Python 3.10 or newer. Standard library only: no dependencies.

Install

pip install docattest                 # the commands docattest and docattest-adapter-python

The core comes inside the wheel: there is one wheel for each of 5 platforms, py3-none-<platform> (the Python code is pure; only the core is native), and pip picks the one built for your machine. Nothing is downloaded when docattest runs, and Node is never needed.

platform name wheel tags
macOS 11+ on Apple silicon darwin-arm64 macosx_11_0_arm64
macOS 10.12+ on Intel darwin-x64 macosx_10_12_x86_64
Linux x86-64, glibc or musl linux-x64 manylinux_2_17_x86_64, manylinux2014_x86_64, musllinux_1_1_x86_64
Linux arm64, glibc or musl linux-arm64 manylinux_2_17_aarch64, manylinux2014_aarch64, musllinux_1_1_aarch64
Windows x64 windows-x64 win_amd64

The name is the platform's in every docattest package: the npm package and the Go archive that carry its core are called by it too.

The Linux core is a static binary, so one wheel serves glibc and musl (Alpine) alike. There is no source distribution, because it could not build the core: on any other platform pip install docattest finds nothing to install, rather than installing a docattest that cannot run. An install from a checkout (pip install -e adapters/python, for development) has no core either; its first run says so, names your platform, and says to install the wheel or set DOCATTEST_CORE.

python3 adapters/python/build_wheel.py --out <dir> --core <platform>=<binary> builds the wheels from cores built for each platform (the release workflow runs it; see CONTRIBUTING.md, "Releasing"). It needs nothing but the standard library and pip, which fetches setuptools for the build.

The launcher looks for the core in three places, in this order, the same as the npm package does:

  1. DOCATTEST_CORE , the path of a core binary. It is asked for its version (--version --json) and refused when it speaks another protocol than this package, or does not answer.

    $ DOCATTEST_CORE=/bin/echo python3 -m docattest check
    docattest: DOCATTEST_CORE (/bin/echo) is not a docattest core: it does not answer --version --json
    [exit 1]
    
  2. docattest/bin/docattest inside this package (docattest.exe on Windows): where the wheel for your platform carries it.

  3. <cache>/core/<version>/<name>/docattest, a cache shared with the npm package, where <name> is the platform's name from the table above. The version is the one in docattest_python/protocol/core.json, which this package pins. <cache> is:

on set cache
any system DOCATTEST_CACHE=/mine /mine
macOS ~/Library/Caches/docattest
Linux XDG_CACHE_HOME=/x /x/docattest
Linux XDG_CACHE_HOME=relative ~/.cache/docattest
Linux ~/.cache/docattest
Windows LOCALAPPDATA=C:\Local C:\Local\docattest
Windows ~\AppData\Local\docattest

XDG_CACHE_HOME counts only when it is an absolute path, as the XDG specification says.

DOCATTEST_TRACE=1 makes the launcher say which core it runs.

From a checkout of the repository, without installing anything:

cargo build --release --manifest-path core/Cargo.toml
export DOCATTEST_CORE="$PWD/core/target/release/docattest"
export PYTHONPATH="$PWD/adapters/python"
python3 -m docattest check --root adapters/python/tests/fixture

docattest hooks install, hooks install --git and agents install write down a command that runs docattest without Node: the virtualenv's own script when the virtualenv is inside the project, else docattest when that is the script the PATH finds, else the script by its full path, else this interpreter by its full path (python -m docattest).

The launcher knows no coding agent. For a virtualenv inside the project it says {root}/.venv/bin/docattest, and the core writes the root as the reader of the command names it: "$CLAUDE_PROJECT_DIR"/.venv/bin/docattest impact --hook in a hook of Claude Code (its descriptor's projectRoot), ./.venv/bin/docattest in AGENTS.md and in a git hook. Set DOCATTEST_COMMAND to have it write something else (uv run docattest).

Any agent, or none, can use the two plain commands, from anywhere inside the project: docattest impact --file <path> after an edit and docattest check before finishing. docattest hooks install --agent gemini (or cursor, or a descriptor file of your own) installs hooks for another agent; docs/agents.md has the details.

A bare core, run without the launcher, finds the adapter as docattest-adapter-python on the PATH, or through DOCATTEST_ADAPTER_PYTHON : a JSON array (["python3", "/path/to/adapters/python/bin/docattest-adapter-python"]) or a command line split on spaces.

Configure

docattest.config.json at the project root:

{
  "pages": ["README.md", "docs/**/*.md"],
  "sources": ["src/**/*.py"],
  "surfaces": { "default": "shoplib" },
  "setup": "docs/docattest_setup.py",
  "adapter": "python"
}
  • surfaces name what pages are allowed to observe. An entry is
    • a module name, "shoplib" or "shoplib.shipping": imported as written. It is looked for in the project root, then in src/, then wherever import finds it;
    • a path, "./src/shoplib" or "./shoplib/shipping.py": that file of the project, imported under the name Python gives it (src/shoplib/shipping.py is shoplib.shipping: the path from the first directory without an __init__.py);
    • both, { "import": "shoplib", "path": "build/lib/shoplib/__init__.py" }: examples import import, and it is read from path.
  • What a surface exports is its __all__ when that is written out as a list. Otherwise it is the module's public names: every function, class and assigned name that does not start with _, and, in a package's __init__.py, what it imports from its own package (from .pricing import total is a re-export; import os is not). uncovered lists the exports no page attests.
  • sources are the Python files whose docstrings hold examples (see below).
  • setup is a Python module with the runners of cases tables and named facts.

A page

This page documents the small shop library that this package's tests use (tests/fixture/), and this guide checks it inside a copy of that project:

# Pricing

A discount is a whole percentage taken off a price.

```python check
from shoplib import Money, apply_discount

assert apply_discount(Money(1000), 25) == Money(750)
```

The same as an interpreter session:

```pycon check
>>> from shoplib import Money, total
>>> total([Money(1000)], "NL")
Money(cents=1210, currency='EUR')
```

Prices are in `EUR` unless told otherwise. <!-- fact: DEFAULT_CURRENCY -->

Tax is known for these regions: <!-- fact: TAX_RATES -->
- `NL`
- `DE`
- `FR`

<!-- cases: discount -->
| case           | price | percent | result |
|----------------|-------|---------|--------|
| a quarter off  | 1000  | 25      | 750    |
| rounds half up | 999   | 50      | 500    |

Discounts above the maximum are refused. <!-- test: tests/test_pricing.py > "TestTotals" > "test_rejects_big_discounts" -->
docattest check          # exit 1 when a page and the code disagree
docattest uncovered      # exports no page attests
docattest where total    # the pages, lines and sentences that attest a symbol
docattest audit          # lists, tables and values that look copied from the code, with no directive
docattest impact         # after an edit: the exports it affects and the prose that names them

A failing proof is reported with its line, what the page expected and what the code gave. The same project has a page that is wrong on purpose, once for each way a proof can fail:

$ docattest check --quiet docs/broken.md
docs/broken.md
  ✗ L5 example: AssertionError: expected Money(cents=700, currency='EUR'), got Money(cents=750, currency='EUR') (block line 3)
  ✗ L11 example: `quote("NL")`: expected 2, got 1 (block line 2)
  ✗ L17 example: cannot import name 'aply_discount' from 'shoplib' (src/shoplib/__init__.py) (block line 1)
  ✗ L21 example: ShopError: No carrier serves the region (block line 2)
  ✗ L26 example: timed out after 3000 ms
  ✗ L31 example: URLError: <urlopen error network access is disabled for attestations; tag the block `check network` to allow it> (block line 2)
  ! L36 example: example at L36: unknown tag "chekc" (known: check, compile, network, continue; add renderer tags to "ignoreTags")
  ✗ L40 fact: expected 60, got 50 from MAX_DISCOUNT
  ✗ L42 fact: expected 2 items from the page, got 3 from TAX_RATES; missing: DE, FR; unexpected: BE
  ! L46 fact: fact `_round(0.5)`: _round is not exported by surface "default" (NameError: name '_round' is not defined)
  ✗ L51 cases: docs/broken.md:51 wrong on purpose: AssertionError: expected '800', got '750'
  ✗ L52 cases: docs/broken.md:52 too much: ShopError: The discount is outside the allowed range
  ! L54 cases: cases runner "nope" at L54 is not in `cases` of docs/docattest_setup.py (known: discount, delivery)
  ! L59 test: tests/test_pricing.py declares no test "test_that_was_renamed"; tests there: "test_discount_rounds_half_up", "TestTotals" > "test_adds_tax", "TestTotals" > "test_rejects_big_discounts"
FAILED
[exit 1]

What each attestation means in Python

Examples: python check, python compile

Blocks tagged python, py, python3, py3 or pycon are the adapter's.

  • check runs the block as a script. The proof is the plain assert statement. A failing assert a == b reports both sides and the line within the block (L5 in the report above); the adapter rewrites comparisons in assert to keep the two values, as pytest does. await may be used at the top level; the block then runs in a new event loop. __name__ is "__main__".

    import asyncio
    
    await asyncio.sleep(0)
    assert __name__ == "__main__"
    

    A block with no assert (and no pytest.raises, self.assert…) runs but is counted as weak.

  • A session is a block tagged pycon , or any block whose first statement is a >>> prompt. It runs the way doctest runs it. Each output must match what the page shows, an expected Traceback must be raised, # doctest: +SKIP and the other doctest directives work, and ... in an expected output matches anything (ELLIPSIS is on; nothing else is). A statement with an output is the session's assertion.

  • compile never runs the block. Python has no compiler to ask, so compile means what can be said honestly from syntax: the block parses and compiles; every import finds its module; a name imported from one of your own modules is bound there (cannot import name 'aply_discount' from 'shoplib'); an attribute read straight off an imported project module exists; and a name the block reads is bound somewhere in it or is a builtin. It is not a type check: a wrong argument or a missing method is not found. check blocks pass through the same test before they run.

  • Isolation. Every block runs in its own process: a fresh __main__, fresh imports of your package, nothing left over from the block before. Blocks of one page share nothing, so each must import what it uses, unless it says continue.

  • continue. Python documentation is often one interpreter session from the top of the page to the bottom: the first block imports, the fifth still uses the import. A block tagged python check continue (or pycon check continue) goes on from the examples before it on its page, back to the nearest one without the word: their statements run first, in the same namespace, and then the block. Of these three blocks, the second goes on from the first and the third starts afresh:

    from shoplib import Money
    
    price = Money(1000)
    assert str(price) == "10.00 EUR"
    
    assert price == Money(1000)
    
    assert "price" not in globals() and "Money" not in globals()
    

    It is still judged alone and still runs in its own process; the earlier blocks are run again for it, and what they prove is their own result, so a session's outputs are compared only in the block that shows them. If an earlier block cannot run, the block fails and says which (the block it continues (L12) did not run: …). compile knows the names the earlier blocks bind, and where credits a block with what it uses of their imports. A first block cannot continue. To bind a page written as one session, tag its first block check and the rest check continue: nothing inside the blocks changes.

  • Renderer attributes in an info string are ignored: ```python title="pricing.py" hl_lines="1 2" check is a check block. Workers are forked from a clean process (os.fork); a block that outlives runtime.timeoutMs has its process group killed. runtime.workers is the number of blocks that run at once.

  • Output. print in attested code goes nowhere; anything written to file descriptor 1 by other means goes to stderr.

  • Network. Internet sockets are refused unless the block is tagged check network (which runs only with DOCATTEST_NETWORK=1). The guard patches socket; an extension module with its own sockets is not stopped.

Docstring examples

Every docstring in a sources file that contains >>> prompts is one example, run as python -m doctest runs it: the session starts with the names of its module, so it needs no import of the thing it documents. The claim it defends is the docstring's description: the text before the first prompt or the first Args:/Parameters/:param section. A function's or class's docstring attests that export; a method's attests its class.

def apply_discount(price: Money, percent: int) -> Money:
    """Take a percentage off a price, rounding half up to a whole cent.

    >>> apply_discount(Money(1000), 25)
    Money(cents=750, currency='EUR')
    """

Facts

A fact expression is a Python expression over the surface's exports:

At most `3` attempts. <!-- fact: RETRY_DEFAULTS["max_attempts"] -->
The rates: <!-- fact column=rate: [f"{r}%" for r in TAX_RATES.values()] -->

It may use the exports, the builtins, snapshot("name"), manifest("path"), the named facts of the setup module, and any standard-library module by its name (math.pi, inspect.signature(f).parameters): a fact cannot import. It may await. A name the surface does not export is an error, which is what keeps a page honest about what a user can see.

manifest("pyproject.toml") is the data of a project file, read and parsed by the core (JSON, TOML and go.mod; the manifests page has the details), as dicts and lists: manifest("pyproject.toml")["project"]["requires-python"]. The commands and the Python version at the top of this page are read so, on any Python the package supports, tomllib or not. The path is relative to the project root and written out in the fact: a path built at run time is never read.

The page's literal is compared with str(value): a value is displayed as its language displays it, and the core knows no spelling of its own. So write values the way Python prints them: True, None, 1.0, ('a', 1). A list on the page is compared with the items of any iterable, each through str. A dict's items are its keys, as iterating it gives them: <!-- fact: TAX_RATES --> above a list checks the keys. A set has no order, so do not combine one with fact ordered.

Cases tables and named facts: the setup module

# docs/docattest_setup.py
from shoplib import ERROR_CODES, Money, apply_discount


def discount(row):                      # one call per row; the row is a dict of strings
    got = apply_discount(Money(int(row["price"])), int(row["percent"]))
    assert str(got.cents) == row["result"]


discount.columns = ["price", "percent", "result"]   # optional: columns the table must have

cases = {"discount": discount}
facts = {"error_messages": lambda: list(ERROR_CODES.values())}

cases maps a runner's name to a function; facts maps a name to something a fact expression may use (<!-- fact: error_messages() -->). A runner may be async. Asserts in the setup module report both sides, like asserts in a block.

Citations

<!-- test: tests/test_pricing.py > "TestTotals" > "test_rejects_big_discounts" --> ties a sentence to a test. A test's title is its function's name: any function or method whose name starts with test, as pytest and unittest find them. Enclosing classes may be named first, outermost first. Citing a class itself is an error that says so ("TestTotals" in tests/test_pricing.py is a class, not a test). The file is read, never run.

Which Python runs your code

The adapter itself only reads syntax. Attested code runs on, in order of preference: the interpreter in DOCATTEST_PYTHON; the active virtualenv (VIRTUAL_ENV); the project's .venv; the interpreter the adapter runs on. So an adapter installed once (with pipx, say) still runs each project's code in that project's environment. DOCATTEST_PYTHON comes first even when the interpreter it names cannot be started:

$ DOCATTEST_PYTHON=/nowhere/python docattest check docs/pricing.md
docattest: cannot start the Python interpreter /nowhere/python that attested code runs on: No such file or directory
[exit 1]
DOCATTEST_PYTHON virtualenv .venv runs on
set active exists DOCATTEST_PYTHON
not set active exists the active virtualenv
not set none exists the project's .venv
not set none none the adapter's own interpreter

What is never your project's own

The adapter tells the core which files and directories to leave alone, wherever they are: __pycache__, site-packages, node_modules, and anything named *.egg-info or *.dist-info. So is any directory that holds a pyvenv.cfg , which is a virtualenv whatever it is called. Hidden directories (.venv, .tox) are left out by the core itself.

Limits

  • No type checking: compile is syntax, imports and names (see above).
  • Exports are read from syntax. An __all__ that is computed at import time is not read; the report warns and the public names are used instead.
  • impact works on top-level functions, classes and assignments. Renaming a function together with its callers reads as a change to all of them.
  • Without os.fork (Windows) every block starts a new interpreter. This path has a test on POSIX but has not been run on Windows.
  • If the adapter is killed with SIGKILL, a block that is still running is not stopped.

Working on this package

CONTRIBUTING.md maps the package's sources, says how to run its tests, lint and type checks, and how this guide is checked.

Metadata

Release files for docattest 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for docattest 0.1.0
File Actions
docattest-0.1.0-py3-none-win_amd64.whl Python 3 none Windows x86-64 Details for docattest-0.1.0-py3-none-win_amd64.whl
docattest-0.1.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl Python 3 none Linux musl 1.1+ x86-64, Linux glibc 2.17+ x86-64 Details for docattest-0.1.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl
docattest-0.1.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl Python 3 none Linux glibc 2.17+ ARM64, Linux musl 1.1+ ARM64 Details for docattest-0.1.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl
docattest-0.1.0-py3-none-macosx_11_0_arm64.whl Python 3 none macOS 11.0+ ARM64 Details for docattest-0.1.0-py3-none-macosx_11_0_arm64.whl
docattest-0.1.0-py3-none-macosx_10_12_x86_64.whl Python 3 none macOS 10.12+ x86-64 Details for docattest-0.1.0-py3-none-macosx_10_12_x86_64.whl

Total release size: 15.4 MB

Release files / docattest-0.1.0-py3-none-win_amd64.whl

Download URL docattest-0.1.0-py3-none-win_amd64.whl
Size 3.2 MB
Tags Python 3 Windows x86-64
SHA-256 checksum
How to use checksums
287fb0b0b84fabb4a42711442cef5ab59e8b1c266ba11a6189a45aaa1ec7ca7e
BLAKE2b-256 checksum
How to use checksums
eb67a45e657bc65378ae2d6a049a62056e93b429bbea7c22512382ad76649e51
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Release files / docattest-0.1.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl

Download URL docattest-0.1.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.musllinux_1_1_x86_64.whl
Size 3.2 MB
Tags Linux glibc 2.17+ x86-64 Linux musl 1.1+ x86-64 Python 3
SHA-256 checksum
How to use checksums
2accec52aeff16a0b7e4edaf249013392bbc10304692c47e26bf736746b15144
BLAKE2b-256 checksum
How to use checksums
1a8e48f0f9c1d37b92d41a9b3e1ae4dce6445e05e74afdf63513fb45503509cc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Release files / docattest-0.1.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl

Download URL docattest-0.1.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl
Size 2.9 MB
Tags Linux glibc 2.17+ ARM64 Linux musl 1.1+ ARM64 Python 3
SHA-256 checksum
How to use checksums
1f16719b1462b1a4dd4ff0aae51d8e9990c5ac62b324dc2b75449add5cc33445
BLAKE2b-256 checksum
How to use checksums
9c2432f39b8e0311cdf6c9ac99a5c0590a9e344869c68f9d51ea181d86e0b795
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Release files / docattest-0.1.0-py3-none-macosx_11_0_arm64.whl

Download URL docattest-0.1.0-py3-none-macosx_11_0_arm64.whl
Size 3.0 MB
Tags Python 3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
b7cb506a92316cb12dd6953f1e3555755982adce5c0b0146a314fe240c83a697
BLAKE2b-256 checksum
How to use checksums
c8c43a251c5c81431454f1dfa5e3f30dae0d15472c15827745029481645b0869
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Release files / docattest-0.1.0-py3-none-macosx_10_12_x86_64.whl

Download URL docattest-0.1.0-py3-none-macosx_10_12_x86_64.whl
Size 3.2 MB
Tags Python 3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
bee8e4ee182c75f77b17e9b4f4b90f044865d1fe16abbffd18a1fcf1fbc81577
BLAKE2b-256 checksum
How to use checksums
c976f95045c8f44e8518c096dababdcf2415abc72fe54697a9a9087c3d6bc490
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

Security for docattest 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 5 files in this release have been uploaded with provenance metadata.

Select from 5 files

Attestation for docattest-0.1.0-py3-none-win_amd64.whl

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 16, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page