llama-index-llms-runcrate

LlamaIndex LLM for Runcrate: 200+ open models (DeepSeek, Qwen, Llama, Kimi, gpt-oss and more) behind one OpenAI-compatible API, billed per token from one prepaid balance. The same account rents GPUs by the hour for fine-tuning or serving your own model.

Install

pip install llama-index-llms-runcrate

Create an API key (docs) and export RUNCRATE_API_KEY=....

Use

from llama_index.llms.runcrate import Runcrate
from llama_index.core.llms import ChatMessage

llm = Runcrate(model="deepseek-ai/DeepSeek-V3.2")

print(llm.complete("Explain LoRA fine-tuning in one sentence."))
print(llm.chat([ChatMessage(role="user", content="Name three uses of an H100.")]))

for chunk in llm.stream_complete("Write a haiku about GPUs."):
    print(chunk.delta, end="")

As the default LLM for RAG and agents

from llama_index.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.agent.workflow import FunctionAgent

Settings.llm = Runcrate(model="Qwen/Qwen3-235B-A22B-Instruct-2507", context_window=131072)

def multiply(a: float, b: float) -> float:
    """Multiply two numbers."""
    return a * b

agent = FunctionAgent(tools=[multiply], llm=Settings.llm)
# response = await agent.run("What is 1234 * 4567?")

Arguments

Other OpenAILike arguments (temperature, max_tokens, timeout, additional_kwargs ...) pass through. Models and prices: runcrate.ai/inference/pricing.

Let your agent rent GPUs too

Remote MCP server https://mcp.runcrate.ai/mcp (OAuth sign-in, no key): setup, client packages aeonmindai/runcrate-mcp.

License

MIT, Aeonmind LLC.