Avadhika

Avadhika is a contract-bound execution runtime. It keeps LLM retries and fallbacks inside one deadline, one admission budget and one attempt policy.

Avadhika (Sanskrit, from avadhi, "limit") means "to the limit; time-bound, of a fixed period".

Status: Level A complete; first public release (0.1.0). The architecture is frozen (docs/spec.md) and the Level A evidence is complete (docs/evidence/LEVEL_A.md). No claim is made beyond the evidence linked below.

The problem

Applications that call several LLM providers wrap each call in retries and fallbacks. Under failure, those layers interact badly:

• Retries stack. A fallback restarts its own retry loop.
• Waits outlive the caller. Backoff sleeps past the caller's deadline.
• Health gets misattributed. A caller's own timeout puts a healthy provider into cooldown.
• Fallbacks go unchecked. A request falls back to a model that cannot take it, with the wrong context window or no tool support.

Afterwards, nobody can say exactly why an attempt was made.

What Avadhika guarantees

A request enters with one monotonic deadline, one hard admission budget and one attempt policy. Every attempt, wait, backoff, provider switch and fallback must fit inside that one contract.

• No attempt the contract can't afford. The runtime never authorizes an attempt whose worst-case cost (all billable dimensions, at a pinned price) or time could break the contract, just because an earlier attempt failed.
• One decision site. Retries and fallbacks are a single PLAN decision against eleven legality predicates. They are not nested loops.
• Failures are attributed. Caller deadlines, transport failures, attempt timeouts and unknown errors never count against a provider's health. Unknown errors stop: no retry, no fallback.
• Retry storms are bounded. A global extra-attempt budget, per-endpoint retry budgets and half-open probe slots are all acquired atomically.
• Streaming commits once. It commits at the first caller-visible delta. After that, output is never spliced from another model.
• Decisions can be replayed. Every decision is recorded with the inputs it consumed, and avadhika replay reproduces it byte for byte. Provider output and network timing are never claimed to be reproducible.

Claim boundary. The budget is a hard admission budget: a limit on what the runtime authorizes. A provider can still bill beyond it, for example for unmodelled dimensions, generation after cancellation, or ignored caps. Those cases are detected and annotated (BILLING_BOUND_VIOLATION), never hidden. See spec section 5.3.

What it is not

• It is not a gateway or proxy.
• It does not pick "the best model". The caller or a policy supplies the candidate order.
• It does no learned routing, load balancing or hedging.

It is a small library with a single runtime dependency: httpx.

Quick start

python -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/avadhika plan     --catalog examples/catalog.json --contract examples/contract.json
.venv/bin/avadhika run      --catalog examples/catalog.json --contract examples/contract.json --prompt hi --record rec.json
.venv/bin/avadhika replay   rec.json
.venv/bin/avadhika explain  rec.json
.venv/bin/avadhika simulate --catalog examples/catalog.json --contract examples/contract.json --scenario examples/scenario-outage.json

The examples use the built-in fake provider and send nothing over the network.

Architecture

Design decisions that interpret the spec are recorded in docs/decisions/. docs/design-overview.md is a reader-oriented edition of the specification; where they differ, docs/spec.md governs.

Evidence

The full Level A evidence report is docs/evidence/LEVEL_A.md.

Using it

• Pass an httpx.AsyncClient. Avadhika's own deadline and attempt timers govern every attempt, and any httpx transport error, timeouts included, is treated as a transport failure. A client-level timeout is redundant, so timeout=None is a reasonable choice.
• Cancel a request with the cancel event or by cancelling its task, for example with asyncio.wait_for. Either way, the attempt is torn down and its lease released before the cancellation propagates.
• Credentialed adapters require an https:// base URL. Local hosts are exempt, and allow_insecure_http=True overrides the check.

Limitations

• Health state, leases and budgets are process-local.
• Admission is conservative. Without a caller-supplied input-token bound, the real adapters fail closed in hard-budget mode.
• A connection-refusing or stalling endpoint never trips its breaker, by design. Its fallbacks draw on the extra-attempt budget, which can run out (scenario S24 measures this).
• If teardown cannot be confirmed, the request is reported as TEARDOWN_UNCONFIRMED with an incomplete decision record, and the endpoint is degraded until teardown is confirmed.
• A failure after a request may have been sent is charged at its full reservation, which can exclude a later fallback under a tight budget. Connect-phase failures are charged nothing.
• required.tools and required.structured_output filter candidates only. The v1 adapters do not send tool or response-format payloads.

License

Apache-2.0