🦜🔗 LangChain × Crusoe AI

This package provides LangChain integrations for Crusoe AI's Managed Inference service, giving you access to leading open-source models powered by Crusoe's proprietary MemoryAlloy™ inference engine.

Features

• OpenAI-compatible API: Drop-in replacement via BaseChatOpenAI
• Leading open-source models: Llama 3.3, DeepSeek V3/R1, Qwen3, Gemma 3, Kimi-K2, GPT-OSS-120B
• Full LangChain support: Streaming, async, tool calling, structured output
• Ultra-low latency: Powered by MemoryAlloy cluster-wide KV cache technology
• LangSmith integration: Built-in tracing and observability

Installation

pip install -U langchain-crusoe

Setup

1. Create an account at Crusoe Cloud
2. Generate an Inference API key in the Security tab
3. Set your API key:

export CRUSOE_API_KEY="your-api-key"

Quick Start

from langchain_crusoe import ChatCrusoe

llm = ChatCrusoe(
    model="meta-llama/Llama-3.3-70B-Instruct",
    temperature=0,
    max_tokens=1024,
)

# Simple invocation
response = llm.invoke("Explain quantum computing in one paragraph.")
print(response.content)

# Streaming
for chunk in llm.stream("Write a haiku about open source."):
    print(chunk.content, end="", flush=True)

Available Models

See the latest model list at the Crusoe Intelligence Foundry.

Advanced Usage

Tool Calling

from pydantic import BaseModel, Field

class GetWeather(BaseModel):
    """Get current weather for a location."""
    location: str = Field(description="City and state, e.g. San Francisco, CA")

llm_with_tools = llm.bind_tools([GetWeather])
response = llm_with_tools.invoke("What's the weather in Seattle?")
print(response.tool_calls)

Structured Output

from pydantic import BaseModel

class Summary(BaseModel):
    title: str
    key_points: list[str]
    sentiment: str

structured_llm = llm.with_structured_output(Summary)
result = structured_llm.invoke("Summarize the benefits of open-source AI models.")

Async Support

import asyncio

async def main():
    response = await llm.ainvoke("Hello, async world!")
    print(response.content)

asyncio.run(main())

Project ID

Optionally set a Crusoe Project ID for usage attribution:

llm = ChatCrusoe(
    model="meta-llama/Llama-3.3-70B-Instruct",
    crusoe_project_id="my-project-id",
)

Or via environment variable:

export CRUSOE_PROJECT_ID="my-project-id"

Configuration

Development

# Clone the repo
git clone https://github.com/crusoecloud/crusoe-developer-hub.git
cd crusoe-developer-hub/integrations/langchain

# Install dependencies
poetry install --with lint,typing,test,test_integration

# Run unit tests
make tests

# Run integration tests (requires CRUSOE_API_KEY)
make integration_tests

# Run linting
make lint

Links

• Crusoe Cloud Console
• Crusoe Serverless Inference Docs
• LangChain Documentation
• API Reference (available after package is published)