LangGraph Agent Toolkit

Serve LangGraph agents through an authenticated HTTP API. Add streaming, persistent conversation history, managed model connections, and optional tools and observability. Use your own client or the included Streamlit interface.

Python 3.11–3.14. The API image uses Python 3.13. Existing deployments should read the migration guide for the current release and changelog before upgrading.

Quickstart · Integrations · Onboarding · Memory and authentication · Deployment · Documentation · Development

What the toolkit provides

• FastAPI routes for invocation, SSE, JSON Lines, conversation lists, history, and feedback.
• Sync and async AgentClient interfaces, including multimodal messages.
• SQLite or PostgreSQL checkpoints with conversation ownership and ordered updates across workers.
• Model factories with connection reuse, timeouts, bounded retries, and an optional aiohttp transport for OpenAI and Azure.
• Native LangChain agents, custom LangGraph workflows, human approvals, and optional Deep Agents.
• Optional tool selection for native agents and Deep Agents. Keep the full catalog by default. Use LLM selection or supported provider tool search.
• MCP tools, Langfuse SDK v2/v3/v4/v5 integration, LangSmith, Phoenix, and local or managed prompts.

Install only the integrations that your service needs. LiteLLM, Langfuse, Phoenix, MCP, Deep Agents, and Streamlit are optional.

Quickstart

This local demo uses a deterministic fake model and SQLite. It needs no model key, Docker, or external service. It checks API behavior and persistence. Model quality requires separate evaluation.

Install uv 0.13.6 or later, then use a fresh checkout:

git clone https://github.com/kryvokhyzha/langgraph-agent-toolkit.git
cd langgraph-agent-toolkit
uv sync --frozen --no-install-project --no-dev --extra uvicorn-backend

Create .env in this fresh checkout with these values. Use a shell without other toolkit or tracing overrides.

USE_FAKE_MODEL=true
AUTH_MODE=trusted
AUTH_SECRET=local-demo-token
AGENT_PATHS=["langgraph_agent_toolkit.agents.blueprints.chatbot.agent:chatbot_agent"]
DEFAULT_AGENT=chatbot-agent
MEMORY_BACKEND=sqlite
SQLITE_DB_PATH=quickstart.sqlite
OBSERVABILITY_BACKEND=empty
MCP_SERVERS={}
MODEL_CONFIGS={}
LANGSMITH_TRACING=false
LANGCHAIN_TRACING_V2=false

Start the API on localhost:

uv run --no-sync python -m langgraph_agent_toolkit.run_api --host 127.0.0.1 --port 8080

In another terminal, check readiness and send a request:

export AUTH_SECRET=local-demo-token

curl --fail http://127.0.0.1:8080/health/ready

curl --fail-with-body http://127.0.0.1:8080/chatbot-agent/invoke \
  -H "Authorization: Bearer ${AUTH_SECRET}" \
  -H 'Content-Type: application/json' \
  -d '{"input":{"message":"Hello"},"user_id":"demo-user","thread_id":"demo-thread"}'

The response content is This is a test response from the fake model. Reuse the same user_id and thread_id for the next turn. The SQLite file retains history after the API stops. Use the demo token only for this local example.

Use the quickstart guide for streaming, saved history, the Python client, the UI, and real-model configuration.

Choose an integration

For an existing Python project, select provider and backend extras:

uv add 'langgraph-agent-toolkit[openai,uvicorn-backend,langfuse-v5]'

Follow the application onboarding guide to register agents, set identity and memory contracts, connect a client, and verify the deployment. It also gives the upgrade sequence for existing applications.

langfuse-v2, langfuse-v3, langfuse-v4, and langfuse-v5 select a Python SDK version range. Choose one. SDK and server versions are separate. SDK v5 requires server 4.59.0 or later. Keep langfuse-v4 or langfuse-v3 for server v3. Use the compatibility guide for supported combinations. mcp, deepagents, ui, and openai-aiohttp are separate extras. See installation for all options.

Memory and authentication

thread_id identifies short-term conversation state. A long-term store can use user_id to identify one user across threads. Passing user_id does not create a store. SQLite supplies checkpoints but has no long-term store.

The service separates conversation storage by authenticated user, agent, and public thread ID. Keep these values consistent when reading or updating history.

Use /threads to list saved conversations for one user and agent. Use /history to read messages in one conversation. History returns at most 100 messages by default. Follow next_offset for more messages and next_cursor for more conversations. See the usage guide for examples and storage limits.

For one deployment per client, keep one AUTH_SECRET. The default AUTH_MODE=trusted preserves 0.9.2 shared bearer-token authentication. user_id is optional. Your trusted application can supply an end user's ID. If it omits the field, the service uses AUTH_SERVICE_USER_ID (default: service). Keep that identity stable for each conversation.

No new auth header or per-user token is required. If an existing deployment sets AUTH_MODE=token, remove that override or set it to trusted to use the compatible behavior. Token mode remains available as an explicit option. AUTH_USERS tokens always identify one user, in either mode. The service rejects a different supplied user ID for those tokens.

Authentication compatibility does not remove the history pagination, checkpoint migration, or validation changes. See the authentication and migration guide for curl examples and existing checkpoint migration.

Deployment

The API can run alone under Uvicorn or Gunicorn. Docker Compose adds the optional frontend, model proxy, and observability services. Configure the full stack through the environment guide.

• Both Docker images use separate build and runtime stages, a slim Python base, and locked production dependencies. See the image build and measurement guide.
• The API Docker image installs and selects aiohttp for managed async OpenAI and Azure calls. Python installations default to HTTPX. Set LLM_HTTP_ASYNC_TRANSPORT=httpx to override the image default.
• Each worker accepts eight active requests by default. Excess work receives 503 before an agent run starts. The optional admission queue is bounded.
• Use PostgreSQL for replicas across hosts. SQLite workers must share a file on a local filesystem. Configure persistence; the default backend is unset.
• Supervisors replace failed workers. Requests running in a failed worker can fail. The toolkit does not provide durable jobs or exactly-once tool writes.

Read deployment for health probes and worker recovery, and reliability for database pools, connection limits, timeouts, cancellation, and retry boundaries.

Documentation

Read the documentation site or the source guides below. The running service exposes its HTTP schema at /docs and /openapi.json.

Development

Install the locked dependencies and run the local tests. The pre-commit hook environments require Python 3.13.

uv sync --frozen --no-install-project --extra all
uv run --no-sync pytest
uv run --no-sync pre-commit run --all-files

Use --extra all because --all-extras selects incompatible Langfuse SDK versions. Ordinary tests use fake models and local services. Process, Docker, PostgreSQL, live Langfuse, and real-model checks have separate setup requirements. Test coverage and local load results do not establish production capacity or model quality.

See CONTRIBUTING.md for the development workflow. Examples are in scripts/python, including the Deep Agents example.

License

MIT.