Aix-Route
A production routing layer that decides when an agent should reason with an LLM and when it should delegate to a tool
Built on the Deterministic Horizon result: extended chain-of-thought stops helping past a measurable depth, and beyond it tool delegation becomes necessary.
Headline findings
| Metric | Value | Why it matters |
|---|---|---|
| Deterministic Horizon $d^*$ | 19–31 steps | Beyond this depth, neural CoT accuracy < 50%. |
| Tool-integrated accuracy | 86–94% | Across 8 task domains and 12 models. |
| Neural CoT accuracy | 24–42% | The same tasks, no tools. |
| Cross-model correlation $r$ | 0.81–0.91 | Models from 6 orgs fail on the same instances ⇒ architectural, not training-specific. |
| Fine-tuning recovery | +3.2% | Theorem 4.10 predicts < 5%; the competing theory predicts > 30%. |
| Cost efficiency (tool vs. CoT) | 4.2–4.7× | Lower cost-per-correct-solution. |
| Decoherence-model fit | R² = 0.96 | Super-exponential decay beats linear (0.71) and exponential (0.83). |
Python API
from deterministic_horizon import (
PermutationTask, generate_instances, evaluate,
estimate_horizon, fit_decoherence_model,
should_delegate, should_delegate_batch, delegation_decision,
horizon_table, recommend_model,
)
# 1. Generate BFS-optimal-depth instances (depth == true BFS optimum)
task = PermutationTask(n_elements=8, seed=42) # S_8, diameter C(8,2)=28
instances = task.generate_instances(n_instances=500, min_depth=5, max_depth=28)
# 2. Evaluate a model (needs an API key in .env)
results = evaluate(model="gpt-4o", instances=instances, conditions=["C1", "C3"])
# 3. Estimate the horizon (super-exponential fit of Theorem 4.2)
horizon = estimate_horizon(results, threshold=0.5)
print(f"d* = {horizon['d_star']:.1f} (R² = {horizon['r_squared']:.3f})")
# 4. Route in your own agent
should_delegate(estimated_depth=horizon['d_star'] + 5, model="gpt-4o") # → True
# 5. Plan a whole decomposition at once, or pick the right model for a depth
should_delegate_batch([5, 8, 35], model="gpt-4o") # → [False, False, True]
recommend_model(estimated_depth=18) # → least over-powered model that still clears 50%
horizon_table() # → per-model d* / ε₀ / L_eff rows (sorted) — the source for `dh horizons`
The five experimental conditions
| Condition | Description |
|---|---|
| C1 | Neural chain-of-thought (standard prompting) |
| C2 | Depth-limited CoT (oracle optimal length) |
| C3 | Tool-integrated (BFS / verifier access) |
| C4 | Length-encouraged prompting ("take as many steps as needed") |
| C5 | Fine-tuned on optimal-length traces |
What's inside
deterministic-horizon/
├── src/
│ ├── policy.py # should_delegate / delegation_decision ← the engineering hook
│ ├── tasks/ # PermutationProbe, FSA-Sim, ArithChain, CircuitTrace, CodeProbe (+ BFS oracle)
│ ├── models/ # Uniform interface: OpenAI / Anthropic / DeepSeek / Gemini / Together / local
│ ├── metrics/ # SSJ, SFE, super-exponential horizon fit, bootstrap CIs
│ ├── analysis.py # Figures + tables (+ plot_model_horizons comparison)
│ ├── runners.py # High-level evaluate(...) Python API
│ └── cli.py # evaluate | analyze | delegate | horizons
├── configs/ # OmegaConf configs (model × task × experiment)
└── tests/ # pytest suite (smoke · metrics · tasks · policy · analysis)
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
aix_route-1.0.1.tar.gz
(75.7 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
aix_route-1.0.1-py3-none-any.whl
(85.0 kB
view details)
File details
Details for the file aix_route-1.0.1.tar.gz.
File metadata
- Download URL: aix_route-1.0.1.tar.gz
- Upload date:
- Size: 75.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ccfe53808672814268758b626e967ed2ad3f7aa94229d780ae4d86dbabffacbc
|
|
| MD5 |
1c0fe8f02c29c8730e493f06fdf8af3e
|
|
| BLAKE2b-256 |
4f9af4d0c7dff0eb08fc8c7f5d816f9852da92260a07cbb4f181aa4a8d553680
|
File details
Details for the file aix_route-1.0.1-py3-none-any.whl.
File metadata
- Download URL: aix_route-1.0.1-py3-none-any.whl
- Upload date:
- Size: 85.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
93b25224970a356a06f72fe730aa75370294b5b1f941f00818ffede2bba524bf
|
|
| MD5 |
5a6f2e0c0ff9527f965bdc63d741a224
|
|
| BLAKE2b-256 |
896422598e590fe29ed6344b5a5829fb6e8eb3dcfcc0cbb636efbc821c3ea1a0
|