LangGraph OpenAI Serve

Serve LangGraph graphs through an OpenAI-compatible /v1 API so existing OpenAI SDKs, Chainlit, Open WebUI, and similar clients can call them without a project-specific protocol.

Install

Use uv for project dependency management:

uv add langgraph-openai-serve

The equivalent pip command is:

pip install langgraph-openai-serve

For deployments that use PostgreSQL for LangGraph checkpoints, Store data, or cross-worker interrupt coordination, install the optional integration:

uv add "langgraph-openai-serve[postgres]"

For polling-only background Responses executed by Hatchet, install its integration. InMemoryBackgroundBackend needs no extra for local trials; see Run Responses In The Background.

uv add "langgraph-openai-serve[hatchet]"

For built-in Langfuse tracing, install the tracing integration:

uv add "langgraph-openai-serve[tracing]"

Set LGOS_ENABLE_LANGFUSE=True together with LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY. LGOS creates the callback lazily on the first graph run; importing the package never initializes Langfuse.

The package contains the OpenAI-compatible server integration, not a built-in LLM graph. Applications register their own graphs. The demo/ checkout keeps each deployable application in an independent uv project with its own lockfile. The entire directory can be copied and run without files from this repository; its lgos-rag example searches the published documentation of the installed LGOS release.

Repository tasks require Bash and Just 1.58.0 or newer. Run just for package recipes and just demo/ for independent demo workflows. Use just --usage <recipe> to see a recipe's options and defaults.

Generate An Application

Create an independent project that lgos serve runs, with example graphs, tests, developer guidance, and Docker:

uvx cookiecutter https://github.com/ilkersigirci/langgraph-openai-serve.git \
  --directory lgos-starter-template

See the starter guide for setup, and Run The LGOS Server to serve your own registry.

Quick Demo

From this repository, prepare the demo environment and PostgreSQL:

cp demo/.env.example demo/.env
just demo/up lgos-db --wait
just demo/api --editable

Then call the demo with the OpenAI Python client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:3004/v1", api_key="DUMMY")

response = client.responses.create(
    model="custom-input-output-context",
    input="Show me custom schemas.",
    store=False,
    safety_identifier="demo-user",
)

print(response.output_text)

Existing Chat Completions clients can call the same simple graph through the same base URL:

completion = client.chat.completions.create(
    model="custom-input-output-context",
    messages=[{"role": "user", "content": "Show me custom schemas."}],
    safety_identifier="demo-user",
)

print(completion.choices[0].message.content)

Use Responses for new clients and advanced workflow features. The Chat example is the compatibility path for existing Chat-only clients.

Use curl http://localhost:3004/v1/models only as a diagnostic to inspect the registered demo graph names.

just demo/api --editable overlays this checkout without changing the self-contained demo project or its lockfile. The demo publishes independent API and Chainlit images and runs a pinned, lightly patched Open WebUI image with the demo's Open WebUI integration mounted into it. See the demo Docker Compose guide.

The complete Compose demo lets one OPENAI_GATEWAY_TYPE=litellm|bifrost setting place either gateway in front of both maintained UI clients. Chainlit and Open WebUI use normal managed/native Responses and Files routes. Metadata comes from LiteLLM's native /model/info after model sync, or Bifrost's native /v1/models after catalog sync. Neither UI connects directly to LGOS. The PostgreSQL-persistent Chainlit client uses a shared mock login by default, with OIDC login available as an opt-in mode. See the Chainlit demo.

Use In FastAPI

from fastapi import FastAPI
from langgraph_openai_serve import GraphConfig, GraphRegistry, LanggraphOpenaiServe
from your_graphs import my_graph

app = FastAPI()
registry = GraphRegistry(
    graphs={
        "my-graph": GraphConfig(
            graph=my_graph,
            description="Answer questions with my LangGraph workflow.",
        )
    }
)

LanggraphOpenaiServe(registry=registry, app=app).bind_openai_api()

The default base URL is {host}/v1. Registered graph names become OpenAI model values.

LGOS accepts native Responses input_file parts with opaque file_id values, but does not own file upload or storage. Deploy one Files API for the graph services that share a file namespace, or use a gateway-native Files provider. The standalone S3-backed demo Files API is a small reference deployment. Chat Completions remains available for direct compatibility clients; the maintained demo UIs use Responses exclusively. See Choose Responses or Chat Completions for a feature-by-feature comparison.

Docs

• Documentation home: docs/index.md
• Package getting started: docs/getting-started.md
• Architecture and state ownership: docs/explanation/architecture.md
• Self-contained demo stack: docs/demo/index.md
• Self-hosted service references: docs/demo/self-hosted.md
• Runnable demo API: docs/demo/api.md
• Demo graph catalog: docs/demo/graphs/
• OpenAI clients: docs/tutorials/openai-clients.md
• Custom graphs: docs/tutorials/custom-graphs.md
• LangGraph runtime settings: docs/how-to-guides/langgraph-runtime-settings.md
• Background Responses: docs/how-to-guides/background-responses.md
• OpenAI-compatible proxies: docs/how-to-guides/openai-proxies.md
• API and configuration: docs/reference.md
• Compatibility contract: docs/explanation/openai-compatibility.md