runcrate-haystack

Haystack chat generator for Runcrate: 200+ open models (DeepSeek, Qwen, Llama, Kimi, gpt-oss and more) behind one OpenAI-compatible API, billed per token from one prepaid balance. The same account rents GPUs by the hour for fine-tuning or serving your own model.

Install

pip install runcrate-haystack

Create an API key (docs) and export RUNCRATE_API_KEY=....

Use

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.runcrate import RuncrateChatGenerator

generator = RuncrateChatGenerator(model="deepseek-ai/DeepSeek-V3.2")
result = generator.run([ChatMessage.from_user("Explain LoRA fine-tuning in one sentence.")])
print(result["replies"][0].text)

Streaming

from haystack.components.generators.utils import print_streaming_chunk

generator = RuncrateChatGenerator(
    model="Qwen/Qwen3-235B-A22B-Instruct-2507",
    streaming_callback=print_streaming_chunk,
)
generator.run([ChatMessage.from_user("Write a haiku about GPUs.")])

In a pipeline

from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder

template = [ChatMessage.from_user("Answer briefly: {{ question }}")]

pipe = Pipeline()
pipe.add_component("prompt", ChatPromptBuilder(template=template, required_variables=["question"]))
pipe.add_component("llm", RuncrateChatGenerator(model="deepseek-ai/DeepSeek-V3.2"))
pipe.connect("prompt.prompt", "llm.messages")

print(pipe.run({"prompt": {"question": "What is an H100?"}})["llm"]["replies"][0].text)

Tool calling

from haystack.components.agents import Agent
from haystack.tools import tool

@tool
def multiply(a: float, b: float) -> float:
    """Multiply two numbers."""
    return a * b

agent = Agent(chat_generator=RuncrateChatGenerator(model="openai/gpt-oss-120b"), tools=[multiply])
print(agent.run([ChatMessage.from_user("What is 1234 * 4567?")])["last_message"].text)

Arguments

The component serializes with to_dict/from_dict, so pipelines that use it save to YAML; the API key is stored as an environment variable reference, never as a value. Models and prices: runcrate.ai/inference/pricing.

Let your agent rent GPUs too

Runcrate also runs a remote MCP server at https://mcp.runcrate.ai/mcp (OAuth sign-in in the browser, no key to paste): setup.

License

MIT, Aeonmind LLC.