asciidoctest 0.2.0a6

= AsciiDoctest: Verifiable, Stateful, and Interactive Documentation with AsciiDoc
:toc: left
:idprefix:
:idseparator: -

image:https://github.com/webmaven/asciidoctest/actions/workflows/ci.yml/badge.svg[CI Status, link=https://github.com/webmaven/asciidoctest/actions/workflows/ci.yml]
image:https://img.shields.io/pypi/v/asciidoctest.svg[PyPI Version, link=https://pypi.org/project/asciidoctest/]
image:https://img.shields.io/badge/python-3.10%2B-blue.svg[Python Version, link=https://www.python.org/]
image:https://img.shields.io/badge/coverage-97%25-green.svg[Test Coverage]
image:https://img.shields.io/badge/License-Apache_2.0-blue.svg[License, link=https://opensource.org/licenses/Apache-2.0]

link:https://github.com/webmaven/asciidoctest[AsciiDoctest] is an executable documentation runner and narrative testing tool designed to parse, collect, and verify link:https://www.python.org/[Python] code blocks directly from link:https://asciidoc.org/[AsciiDoc] (`.adoc`) files and Python docstrings. It is published on link:https://pypi.org/project/asciidoctest/[PyPI].

It integrates AST-based parsing using link:https://github.com/webmaven/asciidoctrine[asciidoctrine] and link:https://github.com/webmaven/asciidocstring[asciidocstring] to verify documentation code examples.

== Why AsciiDoctest?

Extracting doctests using regular expressions can fail when encountering block attributes, nested lists, or complex boundaries.

AsciiDoctest uses **Abstract Syntax Tree (AST)** parsing:

* Parses document structures via link:https://github.com/webmaven/asciidoctrine[asciidoctrine] and link:https://github.com/webmaven/asciidocstring[asciidocstring].
* Honors standard link:https://asciidoc.org/[AsciiDoc] roles, positional arguments, attributes, and includes.
* Maintains source line and column coordinates for failure reporting.

== Executable Documentation State Models

Writing tutorials or reference guides often requires code examples that build on previous steps. At the same time, writers need to run isolated checks without leaking state.

AsciiDoctest provides consistent state models across interactive REPL blocks and script blocks.

A Python code block is considered **marked** when it contains the `test`, `shared`, or `reset` keyword inside its block header—either as a positional argument (e.g., `[source,python,test]`), an explicit attribute (e.g., `[source,python,test="true"]`), or a block role (e.g., `[source,python,role="shared"]`).

Based on these markers, blocks are executed under the following models:

* **No Marker or Attribute**: Treated as a static, non-executable code listing.
+
~~~~
*Exception*: In `eager` mode, unmarked listings are executed as `test` blocks (isolated and ephemeral) ONLY if no block in the document has explicit markers.
~~~~
* **`test` (Isolated & Ephemeral)**: Runs in a clean, isolated namespace (`{}`). Variables or state changes created during execution are discarded.
* **`shared` (Read-Write & Persistent)**: Participates in a continuous, stateful document timeline (like a notebook). Classes, variables, and functions defined in early `shared` blocks are accessible and modifiable in subsequent `shared` blocks.
* **`shared, test` (Ephemeral Copy)**: Receives a snapshot copy of the shared state at that point in the document; mutations or local bindings created within the block are discarded when the block completes.
* **Named Contexts (`[source,python,shared="context_name"]`)**: Maintains separate, persistent state timelines. Blocks specifying the same `shared="<name>"` attribute share state with each other independently of the default shared timeline.
* **Explicit Reset (`[source,python,reset]`)**: Clears accumulated shared state and named contexts, providing a fresh execution environment from that block forward.
* **Section Boundary Scoping**: In multi-section documents, moving across top-level section boundaries (`== Section One` to `== Section Two`) resets the default shared namespace and named contexts. This prevents state leakage between unrelated classes, functions, or topics while preserving sequential flow within each section.
* **Illustrative Includes**: By default, include directives (`include::...[]`) are ignored during doctest extraction (`preprocess_directives=False`), allowing illustrative code references in documentation without requiring referenced files to exist on disk.

This model provides consistent execution across examples and tests documentation directly against your code.

== Features

* **AST-Based Parsing**: Structural parsing using link:https://github.com/webmaven/asciidoctrine[asciidoctrine] (AsciiDoc parser) and link:https://github.com/webmaven/asciidocstring[asciidocstring] (AsciiDoc docstring parser).
* **Execution Modes**:
  - `explicit` (default): Only executes blocks with explicit markers (`test`, `shared`, or `reset`).
  - `eager`: Falls back to executing all `[source,python]` listings as isolated `test` blocks, but only if the document contains zero explicit markers.
* **Section Scoping & Context Management**:
  - Top-level section boundary resets (`==`) isolating document sections.
  - Named context scopes (`[source,python,shared="context_name"]`) for parallel shared state timelines.
  - Explicit reset markers (`[source,python,reset]`) for manual state resets.
* **Direct Python & Docstring Extraction**: Programmatic API (`extract_and_run_docstring_tests`) to extract and run doctests from `.py` files, directories, or loaded modules with per-symbol scope isolation.
* **Tolerant Include Handling**: Illustrative includes are safely skipped during AST extraction (`preprocess_directives=False`), preventing missing file errors for documentation-only references.
* **Pytest Integration**: Automatic discovery and execution of `.adoc` files and Python docstrings via registered link:https://docs.pytest.org/[pytest] collectors.
* **Unittest Compatibility**: Suite wrappers (`DocTestSuite` and `DocFileSuite`) designed to integrate with the standard library link:https://docs.python.org/3/library/unittest.html[unittest] runner.

== Installation

[source,bash]
----
pip install asciidoctest
----

== Pytest Integration

AsciiDoctest automatically registers as a link:https://docs.pytest.org/[pytest] plugin. Simply execute `pytest` in your project directory:

[source,bash]
----
pytest
----

=== Configuration

You can configure collection behavior in your `pyproject.toml` or `pytest.ini`:

[source,ini]
----
[pytest]
asciidoctest_mode = eager
asciidoctest_split_sections = true
----

Alternatively, you can supply command-line flags:

[source,bash]
----
pytest --asciidoctest-mode=eager --asciidoctest-split-sections
----

When `--asciidoctest-split-sections` is enabled, AsciiDoctest collects each test block as an individual pytest item named after its section title and 1-based block index (for example, `README.adoc::Getting_Started::asciidoctest_block_1`), enabling targeted execution via `pytest -k <Pattern>`.

== Unittest Integration

To use the standard library link:https://docs.python.org/3/library/unittest.html[unittest] package, load tests using `DocFileSuite` or `DocTestSuite`:

[source,python]
----
import unittest
from asciidoctest import DocFileSuite

def suite():
    return DocFileSuite("README.adoc")

if __name__ == "__main__":
    unittest.main(defaultTest="suite")
----

== Programmatic API

In addition to pytest and unittest runners, AsciiDoctest provides a programmatic API that runs tests without requiring pytest.

=== `run_adoc_tests` (Unified Runner)

`run_adoc_tests` is the primary entry point for programmatic test execution. It accepts an `.adoc` file path, a `.py` file path, a directory, a loaded `types.ModuleType`, or a raw AsciiDoc content string, and returns a `TestSuiteResult` TypedDict:

[source,python]
----
from asciidoctest import run_adoc_tests

# Run tests in an .adoc file
result = run_adoc_tests("docs/examples.adoc")
print(result)  # => {'total': 4, 'passed': 4, 'failed': 0, 'failures': []}

# Pass setup globals into the execution namespace
result = run_adoc_tests("docs/guide.adoc", setup_globs={"ENGINE": "dag"})

# Collect all failures without stopping at the first one
result = run_adoc_tests("docs/guide.adoc", fail_fast=False)
for failure in result["failures"]:
    print(failure["message"])

# Run against a .py file (routes to docstring extraction)
result = run_adoc_tests("src/mypackage/utils.py")

# Run against a directory
result = run_adoc_tests("src/mypackage/")
----

The returned `TestSuiteResult` contains:

[cols="1,1,3"]
|===
| Key | Type | Description

| `total` | `int` | Total number of test blocks found
| `passed` | `int` | Number of blocks that executed without failure
| `failed` | `int` | Number of blocks that failed
| `failures` | `list[TestFailureDetail]` | Structured failure records (populated when `fail_fast=False`)
|===

Each `TestFailureDetail` entry has: `name`, `message`, `lineno`, `got`, `want`.

=== `extract_and_run_docstring_tests` (Docstring Runner)

`extract_and_run_docstring_tests` (also aliased as `test_docstrings`) discovers and runs AsciiDoc docstring tests directly from Python source files, directories, or imported module objects:

[source,python]
----
from asciidoctest import extract_and_run_docstring_tests, test_docstrings

# Run doctests on a single Python source file
stats = extract_and_run_docstring_tests("src/mypackage/utils.py")
print(stats)  # => {'total': 3, 'passed': 3, 'failed': 0, 'failures': []}

# Run doctests across a directory tree
stats = extract_and_run_docstring_tests("src/mypackage/")

# Run doctests directly on an imported module
import mypackage
stats = test_docstrings(mypackage)

# Collect all failures (default fail_fast=True raises on first failure)
stats = extract_and_run_docstring_tests("src/mypackage/", fail_fast=False)
----

== Testing in Pyodide & WebAssembly

AsciiDoctest and its core dependencies are pure Python and can run in WebAssembly environments like link:https://pyodide.org/[Pyodide].

=== Coexistence with `pytest-pyodide`

AsciiDoctest test items (`AsciiDocItem` and `DocstringTestItem`) initialize `_fixtureinfo` for compatibility with `pytest-pyodide`.

=== Testing Strategies for Pyodide Packages

When documenting and testing frontend or browser-focused Python packages intended for Pyodide:

1. **Pure Pyodide / In-Wasm Execution**: Install `asciidoctest` into the Pyodide runtime (via `micropip.install("asciidoctest")`) and execute tests directly inside Node.js or browser Pyodide using `DocFileSuite`, `DocTestSuite`, or `extract_and_run_docstring_tests`.
2. **Host Pytest Wrapper Bridge**: When orchestrating browser tests from host pytest using `pytest-pyodide` (`@pytest.mark.driver`), parse document blocks on the host using `asciidoctest.parser.parse_adoc_tests(content)` and dispatch block contents into the browser VM using driver helpers such as `selenium.run_python(block.content)`.
3. **Dual-Target Fast Feedback**: Use lightweight mocks for browser globals (`js`, `pyodide.ffi`) during host `pytest` runs for fast local validation, paired with browser integration runs in CI.

=== State Management (`shared` & `reset`) in Pyodide

Frontend and browser-based Python applications maintain state across operations. AsciiDoctest's execution markers map to WebAssembly workflows:

* **Stateful Tutorials (`[source,python,shared]`)**: Re-initializing WebAssembly modules, opening IndexedDB handles, or creating canvas contexts on every snippet is inefficient. Using `shared` allows multi-step browser tutorials where connection objects and state carry sequentially from block to block.
* **Parallel Named Scopes (`shared="<name>"`)**: Isolate distinct browser components or concurrent workers in the same document without collision.
* **Explicit & Section Boundaries (`reset` and `== Section`)**: When transitioning between tutorial chapters or independent widgets, use `[source,python,reset]` or top-level section headings to clear Python shared state.
* **Python State vs. DOM Teardown**: Note that while `reset` and `== Section` clear Python's internal namespace dictionaries, host JavaScript artifacts (such as DOM nodes attached to `document.body` or event listeners added to `window`) persist in the browser engine unless explicitly removed in Python code or test fixtures.

== Examples

Below are standard test blocks demonstrating interactive and script-based execution.

=== Interactive Session

We can run interactive Python sessions with expected outputs. We mark this block with `[source,python,shared]` to allow this interactive block's setup state (`x`) to persist for subsequent blocks:

[source,python,shared]
----
>>> x = "asciidoctest"
>>> x.upper()
'ASCIIDOCTEST'
----

=== Sequential Script Block

We can run script-based test blocks with standard Python assertions. Since they share the same continuous namespace, variables defined in previous `[source,python,shared]` blocks are accessible. We mark this block with `[source,python,shared]`:

[source,python,shared]
----
assert x == "asciidoctest"
y = len(x)
assert y == 12
----

=== Directives Support

Standard `doctest` directives such as `ELLIPSIS` are supported. We mark this block with `[source,python,test]` so that it runs with an isolated namespace:

[source,python,test]
----
>>> print(x)
Traceback (most recent call last):
    ...
NameError: name 'x' is not defined
>>> x = "Hello, beautiful world!"
>>> print(x)
Hello, ... world!
----

=== Ephemeral Copy of Shared State

We can grant a test block access to an ephemeral copy of the accumulated shared state at that point in the document. Modifications made within the block are discarded afterwards. We mark this block with `[source,python,shared,test]`:

[source,python,shared,test]
----
>>> print(x)
asciidoctest
>>> x = "Hello, beautiful world!"
>>> print(x)
Hello, ... world!
----

To demonstrate that the changes in the `shared, test` block were discarded and did not modify the persistent shared namespace, a subsequent `[source,python,shared]` block shows that `x` remains unchanged:

[source,python,shared]
----
>>> print(x)
asciidoctest
----

=== Named Context Scopes

You can maintain multiple distinct, persistent timelines using named context scopes. For example, database and cache states can evolve independently:

[source,python,shared="db_context"]
----
>>> db_conn = {"status": "connected", "database": "users"}
>>> db_conn["status"]
'connected'
----

[source,python,shared="cache_context"]
----
>>> cache = {"user:1": "Alice"}
>>> "db_conn" in dir() or "db_conn" in locals()
False
----

[source,python,shared="db_context"]
----
>>> db_conn["database"]
'users'
>>> "cache" in dir() or "cache" in locals()
False
----

=== Explicit Reset Markers

You can explicitly reset the shared state at any point using the `reset` marker:

[source,python,reset,shared]
----
>>> "x" in dir() or "x" in locals()
False
>>> x = "fresh_start"
>>> x
'fresh_start'
----

== Contributing

We welcome contributions to link:https://github.com/webmaven/asciidoctest[AsciiDoctest]. Please review our link:https://github.com/webmaven/asciidoctest/blob/main/SECURITY.adoc[Security Policy] before running untrusted code blocks during development.

Development guidelines:

* Typed Python checked via link:https://mypy-lang.org/[mypy] in strict mode.
* Code formatting and linting with link:https://github.com/astral-sh/ruff[Ruff].
* Test coverage enforced at **97%** minimum.

To get started on development locally:

1. Clone the repository and install link:https://hatch.pypa.io/[Hatch] with link:https://docs.astral.sh/uv/[uv]:
+
[source,bash]
----
$ git clone https://github.com/webmaven/asciidoctest.git
$ cd asciidoctest
$ pip install hatch uv
----

2. Run the test suite:
+
[source,bash]
----
$ hatch run test
----

3. Check code coverage reports (enforced at ≥ 97%):
+
[source,bash]
----
$ hatch run cov-report
----

4. Run the linter and formatter:
+
[source,bash]
----
$ hatch run lint
$ hatch run format
----

5. Run strict type checking:
+
[source,bash]
----
$ hatch run typecheck
----

6. Build the documentation locally:
+
[source,bash]
----
$ hatch run docs-build
$ hatch run docs-serve
----

== License

Licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at:

link:https://www.apache.org/licenses/LICENSE-2.0[https://www.apache.org/licenses/LICENSE-2.0]

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.