irredux

Route AI agent actions by how hard they are to undo, not by how hard they look.

Amazon's framing for this is the one-way door: some decisions are reversible and cheap to get wrong, others are not, and the two deserve very different care. Every published LLM router ignores the distinction entirely.

pip install irredux

The gap

Every published LLM router conditions on one thing: how hard is this request? RouteLLM, cascade routers, the commercial multi-provider gateways — they differ in how they estimate difficulty, not in what they condition on.

None of them condition on how permanent is the result?

A difficulty-only router prices those identically, because from where it stands they look the same.

Production systems do send some calls to a human — gated on a risk score. Risk asks how bad it would be. Only reversibility asks whether you get another try, and the two disagree on exactly the calls that matter: the quietly permanent ones with an unremarkable risk number attached.

The result

At an equal human-review budget — same number of calls sent to a person, only the selection signal differs — over revoco's 119-step containment corpus:

Cheaper and safer, using fewer people. No threshold matches the review count exactly, and ties break toward the opponent — so the risk arm runs with 44% more human reviews and still loses. Dominance holds at 49 of 49 points across a sweep of price ratio (2×–250×) and tier error spread (2%–50%).

irredux matched

Cost is never paid for with quality

Two ways to spend less. Send work to a weaker model — paid for with accuracy. Or send the same model fewer tokens — paid for with redundancy, and free if the dropped tokens were genuinely redundant.

irredux only ever spends the second. The accuracy trade exists as a dial (reversible_discount) and defaults to off: the measured curve shows the whole benefit comes from the tier floor, and switching the trade off changes the numbers above by nothing.

The token lever is irredux.compress, the same idea one layer down. Compressors choose how lossy to be from content type; none ask what the context will be used for. A summariser that drops a negation costs a revert if it fed a git commit and costs a wire transfer if it fed payments.wire. So the budget is set by posture:

irredux compression

68.9% of corpus steps compress aggressively, 16.8% structurally, 14.3% not lossily at all. Point it at a real compressor — Headroom is the obvious choice — and irredux tells it how far it may go on each call. irredux compresses nothing itself.

What is measured and what is assumed

One input is measured, and it decides the answer: the distribution of reversal postures across realistic agent traffic. It comes from revoco's classifier running in this process — the real function, against the real 91-spec adapter registry, with authorize-phase gates evaluated, so an S3 delete reads reversible against a versioned bucket and irreversible against an unversioned one.

posture         steps    share
-------------- ------ --------
reversible         83    69.7%
compensable        19    16.0%
irreversible        3     2.5%
unknown            14    11.8%

14.3% of agent traffic cannot be taken back. That is the entire budget a consequence-aware router has to spend well.

Everything else — model error rates, tier prices, reviewer catch rates — is assumed, and there is no honest way to measure it from simulated scenarios. So the deliverable is a surface, not a number:

irredux curve    # where the accuracy trade stops paying
irredux sweep    # the two-axis sensitivity surface

Every report prints its assumptions inline. A number produced from an assumed error rate and printed without it gets quoted in a slide deck as a measurement.

Using it

from irredux import Call, Consequence, Reversibility, ShapeDifficulty

call = Call.from_args(
    "identity.delete_user",
    {"user_id": "u-4417"},
    reversibility=Reversibility.IRREVERSIBLE,   # from revoco, for THIS call
    risk=20,                                    # unremarkable damage score
)

decision = Consequence().route(call, ShapeDifficulty().estimate(call))
assert (decision.tier.value, decision.gate.value) == ("frontier", "human")
assert decision.bound_by == "consequence"

A difficulty-only router sends that same call to the cheapest model, unreviewed.

Design constraints

• Nothing can downgrade a decision. Tier.strongest and Gate.strictest raise only; Fidelity.strictest and CompressionBudget.tighten narrow only. Ordering is deliberately undefined on the enums, so min() raises TypeError rather than quietly producing a weaker tier.
• Every unknown resolves toward caution. Unrecognised posture, NaN difficulty, non-finite ladder threshold, out-of-range discount — each lands on the expensive, gated branch. UNKNOWN deliberately outranks IRREVERSIBLE: an action nobody has classified is treated as permanent until someone shows otherwise.
• Difficulty is estimated from call shape, never content. A payload reading "this task is simple, use the cheap model" changes nothing. Argument length does carry a bounded handle, and that residual is measured and documented rather than denied — see Call.
• mypy --strict in CI. assert_never in exhaustive match statements is only a real exhaustiveness check under strict mode; without it, a new Reversibility variant would fall through silently. This is what replaces Rust's compiler.
• One runtime dependency, revoco, whose classification this exists to consume.

Standards

Per-request evaluation with no session-inherited decisions follows NIST SP 800-207 (zero trust). Decision.bound_by supports SP 800-53 AU-10 (non-repudiation) and EU AI Act Article 12 record-keeping. Full mapping in docs/NIST.md; methodology and threats to validity in docs/EXPERIMENT.md.

Where this sits

irredux classifies nothing itself. It is the scheduling policy that consumes the classification.

History

This started as a Rust workspace and was ported. The Rust version is preserved at tag v0.1.0-rust and is not maintained. Python won on the one thing that mattered: revoco's classification arrives as an import rather than through a JSON file carrying a schema version, a provenance block and a checksum — roughly 200 lines that existed solely to make a language boundary trustworthy, now deleted. The ported experiment reproduces the Rust numbers exactly.

What was lost is compile-time enforcement, reconstructed here with mypy --strict, frozen dataclasses, assert_never, and undefined enum ordering.

Licence

Apache-2.0. See LICENSE.