whichmodel-haystack

Haystack integration for WhichModel — cost-aware LLM model selection for your pipelines.

Installation

pip install whichmodel-haystack

Quick Start

from haystack_integrations.components.routers.whichmodel import WhichModelRouter

router = WhichModelRouter()
result = router.run(task_type="code_generation", complexity="high")

print(result["model_id"])       # e.g. "anthropic/claude-sonnet-4"
print(result["provider"])       # e.g. "anthropic"
print(result["confidence"])     # "high", "medium", or "low"

No API key required. The component calls the public WhichModel MCP server at https://whichmodel.dev/mcp.

Usage in a Pipeline

Use WhichModelRouter to dynamically select the best model before generating:

from haystack import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.routers.whichmodel import WhichModelRouter

# Get the best model for the task
router = WhichModelRouter()
result = router.run(
    task_type="code_generation",
    complexity="high",
    estimated_input_tokens=2000,
    estimated_output_tokens=1000,
    budget_per_call=0.01,
    requirements={"tool_calling": True},
)

# Use the recommended model
print(f"Using {result['model_id']} (confidence: {result['confidence']})")
print(f"Estimated cost: ${result['recommendation']['cost_estimate_usd']:.6f}")

Parameters

Init Parameters

Run Parameters

Output

License

Apache-2.0