Skip to main content

stepdoc

stepdoc turns test code into test documentation. From the same pytest tests it writes two documents:

  • a procedure: what the test always does, with symbolic values (dev.map.dac.level = <level>, GET /users/<user_id>), and
  • an executed report: what one run did, with concrete values, pass/fail per step and timing.

Both come from a JSON run record, so they can never drift apart from the code.

Status: 0.2 (alpha). See CHANGELOG.md. SPIKE.md has the design measurements behind it.

Quick look

@pytest.mark.parametrize("name, age", [("Ana", 0), ("Bo", 120)])
def test_user_lifecycle(api, name, age):
    with step("Create user"):
        r = api.post("/users", json={"name": name, "age": age})
        assert r.status_code == 201
        user_id = r.json()["id"]

    with step("Read user back"):
        r = api.get(f"/users/{user_id}")
        assert r.json()["age"] == age
uv run pytest --stepdoc-procedure=out/procedure.md --stepdoc-report=out/report.md

Procedure:

1. Create user
   1.1. POST /users  {"name": <name>, "age": <age>}
   1.2. Verify r.status_code == 201
2. Read user back
   2.1. GET /users/<user_id>
   2.2. Verify r.json()["age"] == age

Report for the Ana, 0 case:

1. Create user   ✓
   1.1. POST /users  {'name': 'Ana', 'age': 0} -> 201
   1.2. Verify r.status_code == 201   ✓
2. Read user back   ✗ AssertionError: assert None == 0
   2.1. GET /users/1 -> 200
   2.2. Verify r.json()["age"] == age   ✗

(Shown as plain text; the Markdown files carry the same content with a parameter table and a summary.)

Run the full example yourself (one case fails on purpose, it is the bug the directed case finds):

uv run pytest examples/quickstart --stepdoc-procedure=out/procedure.md --stepdoc-report=out/report.md

How it works

  1. Steps come from with step("…") blocks (or @step("…") on helpers). Nesting decides the numbering; actions, checks and nested steps share it (1.1, 1.2, 1.2.1).
  2. Actions come from your system under test's own events through a small bridge in conftest.py, for example an httpx event hook or a device's register callbacks. stepdoc ships no drivers.
  3. Symbolic values come from the source: when an action is recorded, stepdoc finds the line of your test that caused it and later reads that expression with executing. Literals stay literal, names become <name>.
  4. Checks come from plain assert (passing ones need enable_assertion_pass_hook = true), step("…", check=...) and stepdoc.attach_check(...). stepdoc never compares values itself.

Keeping documents readable

  • Setup and teardown. Steps run by fixtures go into a Setup section (S1, S2…) and a Teardown section (T1…), apart from the test's own steps.
  • Polling. The same read (or the same request) repeated from one source line, such as while not dev.map.status.pll_locked: ..., becomes one line: Poll dev.map.status.pll_locked, and in the report -> True (37 reads, 18 ms). Turn it off with stepdoc_collapse_repeats = false.
  • Reads inside checks. A read on the same line as an assert is shown on the check's line: Verify dev.map.adc.value == 1024 -> got 1024.
  • Quiet helpers. @step("Reset device", record=False) keeps the step and its checks but drops the actions inside it.
  • Naming a value. stepdoc.value("code", level * 2) returns the value and shows it as <code> in the procedure, also when it travels through a helper first.

Bridges

# conftest.py
def http_bridge(r):
    stepdoc.record_action(
        "{method} {url}", "request", json.loads(r.request.content or "null"),
        value_from=("json",),        # which call argument carries the value
        target_from=("url",),        # template fields resolved from call arguments
        method=r.request.method, url=r.request.url.path, status=r.status_code,
    )

@pytest.fixture
def api():
    with httpx.Client(base_url=URL, event_hooks={"response": [http_bridge]}) as client:
        yield client

@pytest.fixture
def dev(stepdoc_dry_run):
    d = Device.simulated() if stepdoc_dry_run else Device("TCPIP::10.0.0.5")
    d.on_write(lambda reg, val: stepdoc.record_action(reg.path, "write", val, value_from=("value",)))
    d.on_read(lambda reg, val: stepdoc.record_action(reg.path, "read", val))
    return d

record_action(target, op, value, *, direction=None, value_from=None, target_from=(), bridge_frames=1, **meta)

  • op is free text (write, read, request, query, measure, …).
  • direction says which way data flows: in (the test sends value), out (value is a result, such as a read or a measurement) or exchange (a request; put the response in meta, for example status= or result=). Common op names default sensibly; anything else is in.
  • Values, meta and targets are redacted before they are stored: keys such as password, token, authorization and bearer tokens become ***. Extend with stepdoc.configure_redaction(keys=[...], patterns=[...]).

Tell stepdoc which modules are library code, so it looks past them for your line:

[tool.pytest.ini_options]
stepdoc_skip_modules = ["httpx", "mydevice"]
enable_assertion_pass_hook = true

Options

Option Purpose
--stepdoc-record=PATH Write the JSON run record (schema: src/stepdoc/schema/run-record.schema.json)
--stepdoc-procedure=PATH Write the procedure. The extension picks the format: .md, .html (one self-contained file) or .txt
--stepdoc-report=PATH Write the executed report, same formats
--stepdoc-dry-run Collect the procedure without judging: assertion failures inside steps are recorded and skipped so later steps are still documented
stepdoc_skip_modules (ini) Module prefixes skipped when looking for your source line
stepdoc_collapse_repeats (ini) Collapse polling into one line (default true)
stepdoc_adapters (ini) Adapters to run. Default: every adapter whose tool is installed. -verify leaves one out, none runs none

Without any of these options the plugin records nothing and prints nothing.

All options work with pytest-xdist (-n 4): each worker records the tests it runs, and the main process writes one record and one set of documents, with tests in collection order. Tests on a worker that crashes are missing from the documents.

The session-scoped stepdoc_dry_run fixture (or stepdoc.is_dry_run(config)) tells your fixtures to use a simulated backend. Python evaluates arguments eagerly and dry-run only documents the branches the simulation takes.

Adapters

stepdoc works with other pytest plugins without depending on them, and they don't depend on stepdoc. When one of them is installed, its adapter reads what the plugin already publishes:

  • pytest-verifier: each verify.* check becomes a "Verify …" line in the step it was made in, taken from the pytest_verify_results hook. An expected value given by name is symbolic in the procedure (Verify 'Stored age' == <age>) and concrete in the report (== 30 -> got 31 ✗).
  • pytest-strategies: the report shows each case's seed and the row it ran (directed: youngest), and the procedure's parameter table shows what each argument draws from (integer 0 … 120) instead of the sampled values.

If a plugin that judges checks runs with its adapter off, stepdoc can't see those checks, so it shows the steps of the tests that use it as ?, never ✓, next to the test's own verdict. In dry-run, verify's checks are documented as not judged, but verify still judges them, so a failing one still fails the test.

Other packages can ship adapters through the stepdoc.adapters entry-point group (a subclass of stepdoc.adapters.Adapter). Plugins can also follow steps without importing stepdoc, through the pytest_stepdoc_step_started, pytest_stepdoc_step_finished and pytest_stepdoc_record_ready hooks (implement them with @pytest.hookimpl(optionalhook=True)).

Rendering a saved record

stepdoc render turns a run record into documents again, without running the tests or needing pytest installed:

stepdoc render out/run.json --procedure procedure.html --report report.md
stepdoc render out/run.json                          # procedure as Markdown on stdout
stepdoc render out/run.json --doc report --format text

Agent skill

stepdoc ships a skill that teaches AI coding agents (Claude Code, Codex, GitHub Copilot and others) to write tests with stepdoc and review the procedure:

stepdoc skill install            # .claude/skills/ and .agents/skills/ in this repository
stepdoc skill install --claude   # only .claude/skills/ (--agents for only .agents/skills/)
stepdoc skill install --global   # in your home folder instead
stepdoc skill status             # where it is installed, and whether it is outdated
stepdoc skill uninstall

Reinstalling an unchanged copy does nothing; a copy you edited is never overwritten or removed without --force. --print shows the skill instead of writing it.

Without pytest

stepdoc.core never imports pytest, so the same API works in scripts and test stations:

from stepdoc import Recorder, step, dumps

with Recorder("station-42") as rec:
    with step("Configure"):
        ...
open("run.json", "w").write(dumps(rec))

Development

uv sync
uv run pytest
uv run mypy
uv run python bench/bench_events.py      # cost per recorded event
uv run --python 3.11 pytest              # any supported Python (3.11+)

Metadata

Release files for stepdoc 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for stepdoc 0.2.0
File Size Uploaded Provenance Actions
stepdoc-0.2.0.tar.gz 121.2 kB With provenance Details for stepdoc-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for stepdoc 0.2.0
File Interpreter ABI Platform Actions
stepdoc-0.2.0-py3-none-any.whl Python 3 none any Details for stepdoc-0.2.0-py3-none-any.whl

Total release size: 188.1 kB

Release files / stepdoc-0.2.0.tar.gz

Download URL stepdoc-0.2.0.tar.gz
Size 121.2 kB
Tags Source
SHA-256 checksum
How to use checksums
029612be0ae1bf82aa7c55094475b3b6bdf1a9f96cabf2d6801b92a576f1632d
BLAKE2b-256 checksum
How to use checksums
d915d8413f70a3d2b17fe7b6af30f140d11fdc188c8e6b5d702e5697c79a2469
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

Release files / stepdoc-0.2.0-py3-none-any.whl

Download URL stepdoc-0.2.0-py3-none-any.whl
Size 66.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7539d9e1b7d248641bbfe797884fd568540851e2840e58063b91c2dc9fd5c9e2
BLAKE2b-256 checksum
How to use checksums
0792955561099495b91aef401ecaf661e0422e5dfcb0fee48e283b39b8d785f4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

Security for stepdoc 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for stepdoc-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page