🐸 ModelHop

Stop overpaying for AI. Every query hops to the cheapest model that can handle it.

CLI · Python SDK · OpenAI-compatible server — one brain, many bodies 🐸

Asking GPT-4 "what is Python?" is like taking a taxi to your mailbox. ModelHop routes each query to the cheapest model capable of answering it — free models for the easy stuff, premium muscle only when it earns its keep — and shows you exactly what you saved on every single call.

$ modelhop r "What is Python?"

  🎯 Routed to groq-qwen3-8-27b 🆓 FREE

  💸 Cost Analysis
  💰 Actual cost:               $0.0000
  ❌ Would cost (openai-gpt-4):  $0.0010
  ✨ You saved:                 $0.0010 (100%)
  🎉 AMAZING SAVINGS

Start hopping in 60 seconds 🐸

1. Install

pip install modelhop

2. Add one free key — grab one at console.groq.com or aistudio.google.com, then:

modelhop init     # creates modelhop.yaml, then offers to run setup
modelhop setup    # paste your key; broken models get tested alternatives

3. Route your first query

modelhop r "How do I reset my password?"
modelhop stats    # lifetime savings, hop score, latency

That's it. No config editing, no SDK keys to juggle — if a provider renames a model or your key lacks access, setup tells you the real reason and offers a working alternative.

Three ways to hop — pick yours 🐸

CLI — beautiful terminal output with cost panels, hop scores, token counts, and --json for scripts:

modelhop r "Implement quicksort" --verbose   # full routing rationale
modelhop r "Summarize this log" --json       # machine-readable + tokens
modelhop providers                           # live status with real failure reasons

Python SDK — async-first, same brain as the CLI:

import asyncio
from modelhop import ModelHop

async def main():
    mh = ModelHop()
    r = await mh.route(
        "Explain SQL vs NoSQL",
        budget=0.05,                      # hard spend cap for this task
        trust_required={"zdr": True},     # hard privacy constraint
    )
    print(r.response)                     # the answer
    print(r.model, r.cost.savings)        # where it went, what you saved
    print(mh.explain(r))                  # one-line why

asyncio.run(main())

Server — point any OpenAI client at it, change nothing else:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="not-needed-locally")
client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "hi"}])

Streaming, /v1/models, optional Bearer auth (MODELHOP_API_KEY), rate limits and body caps included.

Why frogs hop smarter 🐸

• 🧠 Cheapest capable model, every time — multi-signal analysis (intent, complexity, code cues, constraints) plus experience memory picks the tier; premium models only fire when the query earns it.
• 💰 Savings on every call — actual vs would-have cost, tokens in/out, and a hop score, printed per query and aggregated in stats (lifetime totals, --reset to clear).
• 🛡️ Fail-closed by default — low-confidence answers pin to the safest fallback and say so (degraded - low confidence…, never silent); cascade fallback rides through provider outages.
• 📚 Self-improving — outcomes feed a contextual bandit + adaptive confidence threshold; calibration, shadow→canary rollouts, and drift auto-rollback keep it honest.
• 🔍 Auditable — hash-chained signed ledger, JSONL traces, quality shield, OTel spans. Your keys never touch logs or traces.
• 🤖 Agent-ready — MCP tool hook (mcp_tool_hook), LlamaIndex wrapper (pip install "modelhop[llamaindex]"), task budgets, session pinning, and trust policies.
• 🔌 Honest wiring — setup probes live endpoints and reports the real error (bad key? no credits? model retired?) instead of "check your key", then offers tested replacements.

Providers 🐸

Free tier covers everyday questions; complex reasoning and code escalate to premium automatically. No mid tier? Fallback skips straight from free to premium. OpenAI needs prepaid billing credits before any of its models respond — setup will tell you exactly that.

Commands at a glance 🐸

Piped output stays clean everywhere: no ANSI colors, ASCII frames, and emoji stripped when stdout isn't a terminal (plus NO_COLOR / MODELHOP_PLAIN=1 support).

Documentation 🐸

• How routing works — pipeline, behaviors, CLI reference
• Python SDK — response object, budgets, trust, errors
• Configuration — full modelhop.yaml schema
• Troubleshooting — setup failures, stats, hop score, degraded

Development 🐸

git clone https://github.com/aalok101singh/modelhop.git
cd modelhop
pip install -e ".[dev,server]"

pytest --cov=modelhop --cov-fail-under=60   # 255 tests, hermetic (no live keys needed)
ruff check modelhop/ tests/
black --check modelhop/ tests/

Contributing 🐸

Contributions welcome! Please see CONTRIBUTING.md for guidelines.

License 🐸

MIT License — see LICENSE for details.

Security 🐸

Found a vulnerability? Please see SECURITY.md — don't open a public issue.

Changelog 🐸

See CHANGELOG.md for version history.

🐸 Made with love by Aalok — hop smart, pay less.