pytest-codex

Building toward pytest-native tests for what Codex actually does.

Status: 0.0.1 core preview

This initial release provides working utilities for parsing the JSONL output of codex exec --json and detecting file changes with before/after snapshots. It does not register a pytest plugin yet. The codex fixture, command runner, record/replay, and repeated-run pass-rate checks are under development and are not available in this version. No Codex calls are made by installing or importing this package.

Requires Python 3.10 or newer.

Parse a saved execution trace

from pathlib import Path
from pytest_codex.events import Command, build_items, parse_jsonl, total_usage

events = parse_jsonl(Path("trace.jsonl").read_text(encoding="utf-8"))
for item in build_items(events):
    if isinstance(item, Command):
        print(item.command, item.exit_code)
print(total_usage(events).total)

The parser removes common shell wrappers from command strings and preserves unknown item types as raw data. Token totals count input plus output tokens; cached input tokens are already part of input and are not added again.

Detect file changes

from pathlib import Path
from pytest_codex.snapshot import snapshot, diff

root = Path("my-project")
before = snapshot(root)
# Run your own operation that changes files here.
after = snapshot(root)
print(diff(before, after).to_dict())

Snapshots detect edits performed through shell commands as well as direct file edits. Common generated directories such as .git and .venv are excluded.

Development

Run the core smoke tests from a source checkout:

python -m pip install .
python -m unittest discover -s tests -v

The repository URL is https://github.com/Czj027/pytest-codex. The GitHub repository itself may not be publicly available yet.

MIT licensed. Independent community project; not an official OpenAI package.

中文说明

0.0.1 是有实际功能的核心预览版：包含 Codex JSONL 事件解析和文件变更快照。 pytest fixture、录制/回放和多次运行通过率统计尚未实现，本版本不会注册 pytest 插件。 安装和导入不会调用 Codex，也不会消耗模型额度。