ordeal
Find a real failure. Keep it fixed.
Ordeal scans existing Python code with generated inputs, reproduces failures, and can turn a supported finding into a pytest regression. The first scan needs no test code or configuration.
See it find a bug
Given this function:
# risky.py
def average(values: list[float]) -> float:
"""Return the arithmetic mean."""
return sum(values) / len(values)
Run Ordeal without installing it:
uvx ordeal scan risky.py
The relevant part of the output is:
ordeal scan: risky
status: findings found
evidence cards:
- risky.average [supported]
claim: The recorded input reproducibly makes risky.average raise
ZeroDivisionError: division by zero.
witness: input={"values": []}
replay: verified (2/2 exact matches)
boundary: Same exception type, message, and terminal source location.
next: ordeal scan risky.py --save
supported is deliberately narrow. It means the same input reproduced the same
recorded failure during immediate replay. It does not prove the root cause,
untested behavior, or that a future fix works.
Run it on your project
pip install ordeal # or: uv tool install ordeal
cd your-project
ordeal scan . # auto-detect; write no project artifacts
A normal scan imports and executes target code. Isolate code that can send email, mutate production data, or call live services.
Keep the failure fixed
ordeal scan . --save
uv run pytest tests/test_ordeal_regressions.py -q # prove it fails before the fix
# fix the product code
ordeal verify <finding-id> --allow-unsafe-artifacts
ordeal verify --ci
Commit tests/test_ordeal_regressions.py with
tests/ordeal-regressions.json. The richer .ordeal/findings/ review history
may stay local.
The complete beginner workflow is:
scan → save one witness → fix → verify the same witness → guard it in CI
What the result means
| Result | Meaning | Next action |
|---|---|---|
supported |
The recorded failure matched during immediate replay | Save, fix, verify |
exploratory |
Interesting signal without the same replay support | Investigate; do not call it a proven bug |
expected |
The input violated a known precondition | Usually no product fix |
blocked |
Ordeal could not construct enough of the target | Inspect targets or add a harness |
no findings yet |
Nothing failed in the sampled run | Useful evidence, not a correctness proof |
Each scan also emits a reliability map from source-backed retry, recovery, I/O,
transaction, and ML/data seams to candidate properties. PASS,
NOT EXERCISED, and FAIL describe the observed operation × fault × property
evidence; mined properties remain hypotheses. Use --deepen --time-limit 60
for one safe planned follow-up, or --base-ref origin/main to prioritize
changed operations. Fault probes close only their narrow operational cell after
the named injection boundary is actually reached.
Why Ordeal
- No test boilerplate for the first signal. Point it at a project, module, Python file, or callable.
- Evidence instead of a generic warning. A supported finding binds the source, exact input, exception seam, and replay result.
- A bug can become protection.
--saveproduces the review bundle and, when the witness is renderable, a durable pytest regression.
Start with the Scan Quickstart. If a method needs setup or state, continue to Object Harnesses. The Durable Regression Workflow and CI guide cover the full red-fix-green loop.
Advanced workflows
Use these only when the default scan or your specific goal requires them.
| Goal | Start here |
|---|---|
| Judge whether tests detect real changes | Test Protection |
| Write a custom stateful chaos test | Custom Chaos Tests |
| Exercise long-lived services | Service Evidence Loop and Compose Evidence Loop |
| Put service recovery in CI | Compose CI and operations |
| Compare two functions | Differential Quickstart and Divergence Evidence |
| Compare committed revisions | Revision Diff, troubleshooting, and schema |
| Compare a stateful refactor | System Differential Testing |
| Replace a module without preserving old bugs | Safe Migrations and Migration Workflow |
| Inspect every command or Python type | CLI reference and API reference |
AI coding agents
Ordeal ships with AGENTS.md, which teaches coding agents to start with the same scan-first workflow and keep advanced commands behind an explicit need.
Development
git clone https://github.com/teilomillet/ordeal
cd ordeal
uv sync --locked --extra dev
uv run pytest
See CONTRIBUTING.md and CHANGELOG.md.
License
Apache 2.0
1 maintainer
Metadata
1 maintainer
Release files for ordeal 0.3.43
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| ordeal-0.3.43.tar.gz | 1.0 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| ordeal-0.3.43-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 2.0 MB
Release files / ordeal-0.3.43.tar.gz
| Download URL | ordeal-0.3.43.tar.gz |
|---|---|
| Size | 1.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
fff8b824936397744c611f9f8e6c27bf0d1697a5c8933e5e522aabaf542d2eb4
|
|
BLAKE2b-256 checksum How to use checksums |
af337c69eefca480bafd296cf11e856abe5bca682d2dd5d0e0f7eeb75caba6f1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 11, 2026.
Transparency logRelease files / ordeal-0.3.43-py3-none-any.whl
| Download URL | ordeal-0.3.43-py3-none-any.whl |
|---|---|
| Size | 925.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
785190250d2ee0879e565440bb0b4cee00d02963c86f5ed13fb137bd220f9e9e
|
|
BLAKE2b-256 checksum How to use checksums |
0f2af4b9f4abb350a683d92dacf84d917913630c1e03fc926ef3088e4fb6df61
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 11, 2026.
Transparency log
