clitag

A CLI tester and documentation generator.

clitag runs CLI commands as subprocesses, collects their output, and renders the results as Markdown documentation. It is designed to serve two purposes at once:

1. Testing — exercise your CLI tool's entry-points from pytest and assert on their stdout.
2. Documentation generation — because the steps are run in a real test, the resulting Markdown is always up-to-date. Write it to a file from your test and commit it alongside your code.

Installation

pip install clitag

If you want subprocess coverage tracking (see below):

pip install clitag[coverage]

For development / running the test suite:

pip install clitag[testing]

Basic usage

from clitag import CliTester

ct = CliTester()
ct.title = "My CLI example"
ct.description = "The following steps show typical usage."

output = ct.run_step(
    cmd="echo",
    args=["hello, world"],
    description="Print a greeting",
)
assert "hello" in output

ct.run_step(
    cmd="my-tool-status",
    description="Check the status",
)

print(ct.format_md())

run_step returns the decoded stdout of the process, so you can assert on it directly alongside capturing the documentation.

Writing the Markdown to a file

A common pattern is to write the Markdown from within a pytest test so it is regenerated on every run:

def test_cli_example(tmp_path):
    ct = CliTester()
    ct.title = "Example usage"
    ct.run_step("my-tool-cmd", description="Run the command")

    with open("docs/example_usage.md", "w") as f:
        f.write(ct.format_md())

Coverage convention

Why subprocess coverage is tricky

pytest-cov / coverage.py only tracks the current Python process. When your CLI entry-points are invoked as subprocesses (e.g. my-tool-cmd), coverage is lost for those code paths.

How clitag solves it

When a CLI_PREFIX is configured, clitag rewrites matching commands to:

coverage run --data-file=... --rcfile=... -m <module> <args>

For example, with CLI_PREFIX = "my_tool_" and CLI_PREFIX_REPLACE = "my_tool.cli_", the command my-tool-cmd becomes:

coverage run --data-file=.coverage --rcfile=.coveragerc -m my_tool.cli_cmd

clitag also sets the COVERAGE_PROCESS_START environment variable so that child processes started by the CLI command also report coverage.

Requirements for coverage to work

1. Install coverage: pip install clitag[coverage]

2. A .coveragerc (or [tool.coverage] in pyproject.toml) in your project root with:

   [run]
   parallel = true
   branch = true
   

3. Run coverage combine after the test suite to merge the parallel data files.

Usage patterns

Constructor — ad-hoc:

ct = CliTester(
    cli_prefix="my_tool_",
    cli_prefix_replace="my_tool.cli_",
)

Subclass — recommended for whole-project reuse:

class MyProjectTester(CliTester):
    CLI_PREFIX = "my_tool_"
    CLI_PREFIX_REPLACE = "my_tool.cli_"

Subclassing is preferred when you have many test files that all test the same project, because the prefix convention is defined once and inherited everywhere.

When CLI_PREFIX is None (the default), no coverage wrapping is applied and commands are executed as-is.

pytest fixture

Installing clitag registers a cli_tester pytest fixture automatically via the pytest11 entry-point — no import or conftest declaration is needed:

def test_something(cli_tester):
    output = cli_tester.run_step("echo", args=["hello"])
    assert "hello" in output

The fixture returns a plain CliTester() with no prefix configured. To use a project-specific subclass, override the fixture in your conftest.py:

# conftest.py
import pytest
from clitag import CliTester

class MyTester(CliTester):
    CLI_PREFIX = "my_tool_"
    CLI_PREFIX_REPLACE = "my_tool.cli_"

@pytest.fixture()
def cli_tester():
    return MyTester()

Cross-platform notes

clitag handles the platform difference internally:

• Linux / macOS — uses pexpect.spawn, which supports a full PTY and interactive sendline input.
• Windows — uses pexpect.popen_spawn.PopenSpawn, which does not require a PTY but otherwise behaves the same.

License

MIT — see LICENSE.