ordeal

CI Docs PyPI Python 3.12+ License

Find a real failure. Keep it fixed.

Ordeal scans existing Python code with generated inputs, reproduces failures, and can turn a supported finding into a pytest regression. The first scan needs no test code or configuration.

See it find a bug

Given this function:

# risky.py
def average(values: list[float]) -> float:
    """Return the arithmetic mean."""
    return sum(values) / len(values)

Run Ordeal without installing it:

uvx ordeal scan risky.py

The relevant part of the output is:

ordeal scan: risky
  status: findings found
  evidence cards:
    - risky.average [supported]
      claim: The recorded input reproducibly makes risky.average raise
             ZeroDivisionError: division by zero.
      witness: input={"values": []}
      replay: verified (2/2 exact matches)
      boundary: Same exception type, message, and terminal source location.
  next: ordeal scan risky.py --save

supported is deliberately narrow. It means the same input reproduced the same recorded failure during immediate replay. It does not prove the root cause, untested behavior, or that a future fix works.

Run it on your project

pip install ordeal                  # or: uv tool install ordeal
cd your-project
ordeal scan .                       # auto-detect; write no project artifacts

A normal scan imports and executes target code. Isolate code that can send email, mutate production data, or call live services.

Keep the failure fixed

ordeal scan . --save
uv run pytest tests/test_ordeal_regressions.py -q  # prove it fails before the fix
# fix the product code
ordeal verify <finding-id> --allow-unsafe-artifacts
ordeal verify --ci

Commit tests/test_ordeal_regressions.py with tests/ordeal-regressions.json. The richer .ordeal/findings/ review history may stay local.

The complete beginner workflow is:

scan → save one witness → fix → verify the same witness → guard it in CI

What the result means

Result Meaning Next action
supported The recorded failure matched during immediate replay Save, fix, verify
exploratory Interesting signal without the same replay support Investigate; do not call it a proven bug
expected The input violated a known precondition Usually no product fix
blocked Ordeal could not construct enough of the target Inspect targets or add a harness
no findings yet Nothing failed in the sampled run Useful evidence, not a correctness proof

Each scan also emits a reliability map from source-backed retry, recovery, I/O, transaction, and ML/data seams to candidate properties. PASS, NOT EXERCISED, and FAIL describe the observed operation × fault × property evidence; mined properties remain hypotheses. Use --deepen --time-limit 60 for one safe planned follow-up, or --base-ref origin/main to prioritize changed operations. Fault probes close only their narrow operational cell after the named injection boundary is actually reached.

Why Ordeal

  • No test boilerplate for the first signal. Point it at a project, module, Python file, or callable.
  • Evidence instead of a generic warning. A supported finding binds the source, exact input, exception seam, and replay result.
  • A bug can become protection. --save produces the review bundle and, when the witness is renderable, a durable pytest regression.

Start with the Scan Quickstart. If a method needs setup or state, continue to Object Harnesses. The Durable Regression Workflow and CI guide cover the full red-fix-green loop.

Advanced workflows

Use these only when the default scan or your specific goal requires them.

Goal Start here
Judge whether tests detect real changes Test Protection
Write a custom stateful chaos test Custom Chaos Tests
Exercise long-lived services Service Evidence Loop and Compose Evidence Loop
Put service recovery in CI Compose CI and operations
Compare two functions Differential Quickstart and Divergence Evidence
Compare committed revisions Revision Diff, troubleshooting, and schema
Compare a stateful refactor System Differential Testing
Replace a module without preserving old bugs Safe Migrations and Migration Workflow
Inspect every command or Python type CLI reference and API reference

AI coding agents

Ordeal ships with AGENTS.md, which teaches coding agents to start with the same scan-first workflow and keep advanced commands behind an explicit need.

Development

git clone https://github.com/teilomillet/ordeal
cd ordeal
uv sync --locked --extra dev
uv run pytest

See CONTRIBUTING.md and CHANGELOG.md.

License

Apache 2.0

Project links

Data verified by PyPI on Jul 11, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for teilo from gravatar.com teilo

GitHub Statistics

Data verified by PyPI on Jul 11, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.12

Provides Extra

atheris all dev docs

Tags

testing chaos-testing fault-injection property-testing fuzzing hypothesis chaos-engineering mutation-testing pytest-plugin coverage-guided stateful-testing resilience-testing test-quality

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ordeal-0.3.43.tar.gz (1.0 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ordeal-0.3.43-py3-none-any.whl (925.7 kB view details)

Uploaded Python 3

File details

Details for the file ordeal-0.3.43.tar.gz.

File metadata

  • Download URL: ordeal-0.3.43.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for ordeal-0.3.43.tar.gz
Algorithm Hash digest
SHA256 fff8b824936397744c611f9f8e6c27bf0d1697a5c8933e5e522aabaf542d2eb4
MD5 c1237a718d96ef4917e9230bd676c5d1
BLAKE2b-256 af337c69eefca480bafd296cf11e856abe5bca682d2dd5d0e0f7eeb75caba6f1

See more details on using hashes here.

Provenance

The following attestation bundles were made for ordeal-0.3.43.tar.gz:

Publisher: ci.yml on teilomillet/ordeal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ordeal-0.3.43-py3-none-any.whl.

File metadata

  • Download URL: ordeal-0.3.43-py3-none-any.whl
  • Upload date:
  • Size: 925.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for ordeal-0.3.43-py3-none-any.whl
Algorithm Hash digest
SHA256 785190250d2ee0879e565440bb0b4cee00d02963c86f5ed13fb137bd220f9e9e
MD5 f5ee21096addf40a845c300c66525e52
BLAKE2b-256 0f2af4b9f4abb350a683d92dacf84d917913630c1e03fc926ef3088e4fb6df61

See more details on using hashes here.

Provenance

The following attestation bundles were made for ordeal-0.3.43-py3-none-any.whl:

Publisher: ci.yml on teilomillet/ordeal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.3.43 This release

2 files

0.3.42

2 files

0.3.41

2 files

0.3.40

2 files

0.3.39

2 files

0.3.38

2 files

0.3.37

2 files

0.3.36

2 files

0.3.35

2 files

0.3.34

2 files

0.3.33

2 files

0.3.32

2 files

0.3.31

2 files

0.3.30

2 files

0.3.29

2 files

0.3.28

2 files

0.3.27

2 files

0.3.26

2 files

0.3.25

2 files

0.3.24

2 files

0.3.23

2 files

0.3.22

2 files

0.3.21

2 files

0.3.20

2 files

0.3.19

2 files

0.3.18

2 files

0.3.17

2 files

0.3.16

2 files

0.3.15

2 files

0.3.14

2 files

0.3.13

2 files

0.3.12

2 files

0.3.11

2 files

0.3.10

2 files

0.3.9

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.104

2 files

0.2.103

2 files

0.2.102

2 files

0.2.101

2 files

0.2.100

2 files

0.2.99

2 files

0.2.98

2 files

0.2.97

2 files

0.2.96

2 files

0.2.95

2 files

0.2.94

2 files

0.2.93

2 files

0.2.92

2 files

0.2.91

2 files

0.2.90

2 files

0.2.89

2 files

0.2.88

2 files

0.2.87

2 files

0.2.86

2 files

0.2.85

2 files

0.2.84

2 files

0.2.83

2 files

0.2.82

2 files

0.2.81

2 files

0.2.80

2 files

0.2.79

2 files

0.2.78

2 files

0.2.77

2 files

0.2.76

2 files

0.2.75

2 files

0.2.74

2 files

0.2.73

2 files

0.2.72

2 files

0.2.71

2 files

0.2.70

2 files

0.2.69

2 files

0.2.68

2 files

0.2.67

2 files

0.2.66

2 files

0.2.65

2 files

0.2.64

2 files

0.2.63

2 files

0.2.62

2 files

0.2.61

2 files

0.2.60

2 files

0.2.59

2 files

0.2.58

2 files

0.2.57

2 files

0.2.56

2 files

0.2.55

2 files

0.2.54

2 files

0.2.53

2 files

0.2.51

2 files

0.2.50

2 files

0.2.49

2 files

0.2.48

2 files

0.2.47

2 files

0.2.46

2 files

0.2.45

2 files

0.2.44

2 files

0.2.43

2 files

0.2.42

2 files

0.2.41

2 files

0.2.40

2 files

0.2.39

2 files

0.2.38

2 files

0.2.37

2 files

0.2.36

2 files

0.2.35

2 files

0.2.34

2 files

0.2.33

2 files

0.2.32

2 files

0.2.31

2 files

0.2.30

2 files

0.2.29

2 files

0.2.28

2 files

0.2.27

2 files

0.2.26

2 files

0.2.25

2 files

0.2.24

2 files

0.2.23

2 files

0.2.22

2 files

0.2.21

2 files

0.2.20

2 files

0.2.19

2 files

0.2.18

2 files

0.2.17

2 files

0.2.16

2 files

0.2.15

2 files

0.2.14

2 files

0.2.13

2 files

0.2.12

2 files

0.2.11

2 files

0.2.10

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.1.37

2 files

0.1.36

2 files

0.1.35

2 files

0.1.34

2 files

0.1.33

2 files

0.1.32

2 files

0.1.31

2 files

0.1.30

2 files

0.1.29

2 files

0.1.28

2 files

0.1.27

2 files

0.1.26

2 files

0.1.25

2 files

0.1.24

2 files

0.1.23

2 files

0.1.22

2 files

0.1.21

2 files

0.1.20

2 files

0.1.19

2 files

0.1.18

2 files

0.1.17

2 files

0.1.16

2 files

0.1.15

2 files

0.1.14

2 files

0.1.13

2 files

0.1.12

2 files

0.1.11

2 files

0.1.10

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.0

2 files

0.0.1

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page