ordeal

CI Docs PyPI Python 3.12+ License

Find a real failure. Keep it fixed.

Ordeal scans existing Python code with generated inputs, reproduces failures, and can turn a supported finding into a pytest regression. The first scan needs no test code or configuration.

See it find a bug

Given this function:

# risky.py
def average(values: list[float]) -> float:
    """Return the arithmetic mean."""
    return sum(values) / len(values)

Run Ordeal without installing it:

uvx ordeal scan risky.py

The relevant part of the output is:

ordeal scan: risky
  status: findings found
  evidence cards:
    - risky.average [supported]
      claim: The recorded input reproducibly makes risky.average raise
             ZeroDivisionError: division by zero.
      witness: input={"values": []}
      replay: verified (2/2 exact matches)
      boundary: Same exception type, message, and terminal source location.
  next: ordeal scan risky.py --save

supported is deliberately narrow. It means the same input reproduced the same recorded failure during immediate replay. It does not prove the root cause, untested behavior, or that a future fix works.

Run it on your project

pip install ordeal                  # or: uv tool install ordeal
cd your-project
ordeal scan .                       # auto-detect; write no project artifacts

A normal scan imports and executes target code. Isolate code that can send email, mutate production data, or call live services.

Keep the failure fixed

ordeal scan . --save
uv run pytest tests/test_ordeal_regressions.py -q  # prove it fails before the fix
# fix the product code
ordeal verify <finding-id> --allow-unsafe-artifacts
ordeal verify --ci

Commit tests/test_ordeal_regressions.py with tests/ordeal-regressions.json. The richer .ordeal/findings/ review history may stay local.

The complete beginner workflow is:

scan → save one witness → fix → verify the same witness → guard it in CI

What the result means

Result Meaning Next action
supported The recorded failure matched during immediate replay Save, fix, verify
exploratory Interesting signal without the same replay support Investigate; do not call it a proven bug
expected The input violated a known precondition Usually no product fix
blocked Ordeal could not construct enough of the target Inspect targets or add a harness
no findings yet Nothing failed in the sampled run Useful evidence, not a correctness proof

Each scan also emits a reliability map from source-backed retry, recovery, I/O, transaction, and ML/data seams to candidate properties. PASS, NOT EXERCISED, and FAIL describe the observed operation × fault × property evidence; mined properties remain hypotheses. Use --deepen --time-limit 60 for one safe planned follow-up, or --base-ref origin/main to prioritize changed operations. Fault probes close only their narrow operational cell after the named injection boundary is actually reached.

Why Ordeal

  • No test boilerplate for the first signal. Point it at a project, module, Python file, or callable.
  • Evidence instead of a generic warning. A supported finding binds the source, exact input, exception seam, and replay result.
  • A bug can become protection. --save produces the review bundle and, when the witness is renderable, a durable pytest regression.

Start with the Scan Quickstart. If a method needs setup or state, continue to Object Harnesses. The Durable Regression Workflow and CI guide cover the full red-fix-green loop.

Advanced workflows

Use these only when the default scan or your specific goal requires them.

Goal Start here
Judge whether tests detect real changes Test Protection
Write a custom stateful chaos test Custom Chaos Tests
Exercise long-lived services Service Evidence Loop and Compose Evidence Loop
Put service recovery in CI Compose CI and operations
Compare two functions Differential Quickstart and Divergence Evidence
Compare committed revisions Revision Diff, troubleshooting, and schema
Compare a stateful refactor System Differential Testing
Replace a module without preserving old bugs Safe Migrations and Migration Workflow
Inspect every command or Python type CLI reference and API reference

AI coding agents

Ordeal ships with AGENTS.md, which teaches coding agents to start with the same scan-first workflow and keep advanced commands behind an explicit need.

Development

git clone https://github.com/teilomillet/ordeal
cd ordeal
uv sync --locked --extra dev
uv run pytest

See CONTRIBUTING.md and CHANGELOG.md.

License

Apache 2.0

Project links

Data verified by PyPI on Jul 10, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.3.43)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for teilo from gravatar.com teilo

GitHub Statistics

Data verified by PyPI on Jul 10, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.12

Provides Extra

atheris all dev docs

Tags

testing chaos-testing fault-injection property-testing fuzzing hypothesis chaos-engineering mutation-testing pytest-plugin coverage-guided stateful-testing resilience-testing test-quality

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ordeal-0.3.42.tar.gz (886.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ordeal-0.3.42-py3-none-any.whl (649.9 kB view details)

Uploaded Python 3

File details

Details for the file ordeal-0.3.42.tar.gz.

File metadata

  • Download URL: ordeal-0.3.42.tar.gz
  • Upload date:
  • Size: 886.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for ordeal-0.3.42.tar.gz
Algorithm Hash digest
SHA256 bb87873e62986a7a215c6fd9f4fb3a1ff151bb5e566a58023b2a5e17047ee3ba
MD5 543f4ba624b9d513731f88b28ff22fad
BLAKE2b-256 5195c308be2b7fcfda01a6206f8a71686d081f4c1d68b30dac4595c0b2eb9725

See more details on using hashes here.

Provenance

The following attestation bundles were made for ordeal-0.3.42.tar.gz:

Publisher: ci.yml on teilomillet/ordeal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ordeal-0.3.42-py3-none-any.whl.

File metadata

  • Download URL: ordeal-0.3.42-py3-none-any.whl
  • Upload date:
  • Size: 649.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for ordeal-0.3.42-py3-none-any.whl
Algorithm Hash digest
SHA256 b64622344d43bc9a5294f58e8531c2eb9d4064b9c661c02039162e7ef323edb9
MD5 514b3ad3977b4d4fbdc42576ee02ee69
BLAKE2b-256 83fc25be9de86ae59e416b7a4f0b77abdecbcc429d75f2622ad88f2643e69a4f

See more details on using hashes here.

Provenance

The following attestation bundles were made for ordeal-0.3.42-py3-none-any.whl:

Publisher: ci.yml on teilomillet/ordeal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.3.43

2 files

This release

0.3.42 This release

2 files

0.3.41

2 files

0.3.40

2 files

0.3.39

2 files

0.3.38

2 files

0.3.37

2 files

0.3.36

2 files

0.3.35

2 files

0.3.34

2 files

0.3.33

2 files

0.3.32

2 files

0.3.31

2 files

0.3.30

2 files

0.3.29

2 files

0.3.28

2 files

0.3.27

2 files

0.3.26

2 files

0.3.25

2 files

0.3.24

2 files

0.3.23

2 files

0.3.22

2 files

0.3.21

2 files

0.3.20

2 files

0.3.19

2 files

0.3.18

2 files

0.3.17

2 files

0.3.16

2 files

0.3.15

2 files

0.3.14

2 files

0.3.13

2 files

0.3.12

2 files

0.3.11

2 files

0.3.10

2 files

0.3.9

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.104

2 files

0.2.103

2 files

0.2.102

2 files

0.2.101

2 files

0.2.100

2 files

0.2.99

2 files

0.2.98

2 files

0.2.97

2 files

0.2.96

2 files

0.2.95

2 files

0.2.94

2 files

0.2.93

2 files

0.2.92

2 files

0.2.91

2 files

0.2.90

2 files

0.2.89

2 files

0.2.88

2 files

0.2.87

2 files

0.2.86

2 files

0.2.85

2 files

0.2.84

2 files

0.2.83

2 files

0.2.82

2 files

0.2.81

2 files

0.2.80

2 files

0.2.79

2 files

0.2.78

2 files

0.2.77

2 files

0.2.76

2 files

0.2.75

2 files

0.2.74

2 files

0.2.73

2 files

0.2.72

2 files

0.2.71

2 files

0.2.70

2 files

0.2.69

2 files

0.2.68

2 files

0.2.67

2 files

0.2.66

2 files

0.2.65

2 files

0.2.64

2 files

0.2.63

2 files

0.2.62

2 files

0.2.61

2 files

0.2.60

2 files

0.2.59

2 files

0.2.58

2 files

0.2.57

2 files

0.2.56

2 files

0.2.55

2 files

0.2.54

2 files

0.2.53

2 files

0.2.51

2 files

0.2.50

2 files

0.2.49

2 files

0.2.48

2 files

0.2.47

2 files

0.2.46

2 files

0.2.45

2 files

0.2.44

2 files

0.2.43

2 files

0.2.42

2 files

0.2.41

2 files

0.2.40

2 files

0.2.39

2 files

0.2.38

2 files

0.2.37

2 files

0.2.36

2 files

0.2.35

2 files

0.2.34

2 files

0.2.33

2 files

0.2.32

2 files

0.2.31

2 files

0.2.30

2 files

0.2.29

2 files

0.2.28

2 files

0.2.27

2 files

0.2.26

2 files

0.2.25

2 files

0.2.24

2 files

0.2.23

2 files

0.2.22

2 files

0.2.21

2 files

0.2.20

2 files

0.2.19

2 files

0.2.18

2 files

0.2.17

2 files

0.2.16

2 files

0.2.15

2 files

0.2.14

2 files

0.2.13

2 files

0.2.12

2 files

0.2.11

2 files

0.2.10

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.1.37

2 files

0.1.36

2 files

0.1.35

2 files

0.1.34

2 files

0.1.33

2 files

0.1.32

2 files

0.1.31

2 files

0.1.30

2 files

0.1.29

2 files

0.1.28

2 files

0.1.27

2 files

0.1.26

2 files

0.1.25

2 files

0.1.24

2 files

0.1.23

2 files

0.1.22

2 files

0.1.21

2 files

0.1.20

2 files

0.1.19

2 files

0.1.18

2 files

0.1.17

2 files

0.1.16

2 files

0.1.15

2 files

0.1.14

2 files

0.1.13

2 files

0.1.12

2 files

0.1.11

2 files

0.1.10

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.0

2 files

0.0.1

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page