Llama Stack

Quick Start | Documentation | OpenAI API Compatibility | Discord

Open-source agentic API server for building AI applications. OpenAI-compatible. Any model, any infrastructure.

Llama Stack Architecture

Llama Stack is a drop-in replacement for the OpenAI API that you can run anywhere — your laptop, your datacenter, or the cloud. Use any OpenAI-compatible client or agentic framework. Swap between Llama, GPT, Gemini, Mistral, or any model without changing your application code.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8321/v1", api_key="fake")
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello"}],
)

What you get

Chat Completions & Embeddings — standard /v1/chat/completions, /v1/completions, and /v1/embeddings endpoints, compatible with any OpenAI client
Responses API — server-side agentic orchestration with tool calling, MCP server integration, and built-in file search (RAG) in a single API call (learn more)
Vector Stores & Files — /v1/vector_stores and /v1/files for managed document storage and search
Batches — /v1/batches for offline batch processing
Open Responses conformant — the Responses API implementation passes the Open Responses conformance test suite

Use any model, use any infrastructure

Llama Stack has a pluggable provider architecture. Develop locally with Ollama, deploy to production with vLLM, or connect to a managed service — the API stays the same.

See the provider documentation for the full list.

Get started

Install and run a Llama Stack server:

# One-line install
curl -LsSf https://github.com/llamastack/llama-stack/raw/main/scripts/install.sh | bash

# Or install via uv
uv pip install llama-stack

# Start the server (uses the starter distribution with Ollama)
llama stack run

Then connect with any OpenAI client — Python, TypeScript, curl, or any framework that speaks the OpenAI API.

See the Quick Start guide for detailed setup.

Resources

Documentation — full reference
OpenAI API Compatibility — endpoint coverage and provider matrix
Getting Started Notebook — text and vision inference walkthrough
Contributing — how to contribute

Client SDKs:

Language	SDK	Package
Python	llama-stack-client-python
TypeScript	llama-stack-client-typescript

Community

We hold regular community calls every Thursday at 09:00 AM PST — see the Community Event on Discord for details.

Thanks to all our amazing contributors!

Release files for llama-stack 0.7.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llama-stack 0.7.3
File	Size	Uploaded
llama_stack-0.7.3.tar.gz	15.9 MB	Jul 27, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for llama-stack 0.7.3
File	Interpreter	ABI	Platform	Reset
llama_stack-0.7.3-py3-none-any.whl 782.7 kB Jul 27, 2026	Python 3	none	any	Details

Total release size: 16.6 MB

Release files / llama_stack-0.7.3.tar.gz

Download URL	llama_stack-0.7.3.tar.gz
Size	15.9 MB
Tags	Source
SHA-256 checksum How to use checksums	`5b1f04ec6758c88ab308cee1683bf6e2429aefc151cabd94fb05cc3a1d6ced63`
BLAKE2b-256 checksum How to use checksums	`c56ae948e6366612390901913a21423c121a4ab0a58fd3bc35819bcdcbf7676f`
Upload date	Jul 27, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ogx-ai/ogx
Publishing commit: github.com/ogx-ai/ogx/tree/5bc3e40f5f2518fd38f1c24d0b2a273108f8b55f

Workflow

Publishing configuration: github.com/ogx-ai/ogx/blob/5bc3e40f5f2518fd38f1c24d0b2a273108f8b55f/.github/workflows/pypi.yml
Publishing logs: github.com/ogx-ai/ogx/actions/runs/30275261348/attempts/1

Artifact

Subject: llama_stack-0.7.3.tar.gz
SHA-256 checksum: 5b1f04ec6758c88ab308cee1683bf6e2429aefc151cabd94fb05cc3a1d6ced63 Verifying attestations

Release files / llama_stack-0.7.3-py3-none-any.whl

Download URL	llama_stack-0.7.3-py3-none-any.whl
Size	782.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`c7b3e07a6f45a5db0a7d8779c5008b347db5b1b33fce71078e8904590bf71263`
BLAKE2b-256 checksum How to use checksums	`e2a5932578511d653095649287161642993a4896db24480053086a024f382f93`
Upload date	Jul 27, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/6.1.0 CPython/3.13.7`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/ogx-ai/ogx
Publishing commit: github.com/ogx-ai/ogx/tree/5bc3e40f5f2518fd38f1c24d0b2a273108f8b55f

Workflow

Publishing configuration: github.com/ogx-ai/ogx/blob/5bc3e40f5f2518fd38f1c24d0b2a273108f8b55f/.github/workflows/pypi.yml
Publishing logs: github.com/ogx-ai/ogx/actions/runs/30275261348/attempts/1

Artifact

Subject: llama_stack-0.7.3-py3-none-any.whl
SHA-256 checksum: c7b3e07a6f45a5db0a7d8779c5008b347db5b1b33fce71078e8904590bf71263 Verifying attestations