pydantic-rag-agent

A retrieval-augmented generation (RAG) agent built on Pydantic and managed with uv.

Status: M0–M7 are complete and published as 0.3.0; M8 is in progress (citations download their originals). Ingestion, retrieval, the agent, the API, the CLI, the Streamlit UI and the OpenAI-compatible endpoint all work; the suite runs offline at 92% branch coverage. OPS-1 (instrumentation) is deliberately out of scope. main is the state described in docs/overview.md; the ordered plan behind it is in docs/features/ and docs/roadmap.md.

Why uv?

Dependency resolution and virtualenv handling live in one fast tool, and uv.lock pins the exact graph so your CI and your laptop agree on what "it works" means.

Documentation

Full documentation lives in docs/ — start with the documentation index or the project overview. Design rationale is recorded as decision records.

Coding agents should read AGENTS.md first.

Requirements

• uv 0.13 or newer
• Python 3.11+ — uv will fetch and manage the interpreter for you if needed

Install

uv add "pydantic-rag-agent"               # core: agent, loaders, CLI, in-memory store
uv add "pydantic-rag-agent[chroma]"       # plus the default persistent store
uv add "pydantic-rag-agent[chroma,ui]"    # plus the Streamlit UI
uv add "pydantic-rag-agent[chroma,api]"   # plus the FastAPI integration
uv add "pydantic-rag-agent[chroma,s3]"    # plus originals in a bucket
uv add "pydantic-rag-agent[all]"          # everything

The core install has no vector database: it runs with the in-memory store, which is fine for a script that ingests and asks in one go. Anything the agent does not need at import time is an extra, and a missing one tells you which to install.

Published as 0.3.0 — see the changelog for what is in it. Before 1.0 the public API may change in a minor release, with a migration note.

The ways to use it

There are six ways in, and they differ in where the agent lives rather than in what it can do. Pick by asking who owns the process:

All six share one façade, one configuration tree and one set of citation semantics. Nothing below is a different feature set — only a different owner of the event loop.

1. Library

The agent runs in your process. No server, no port.

from pydantic_rag_agent import RagAgent

agent = RagAgent.from_env()  # reads PRAGENT_* / OPENROUTER_API_KEY
agent.ingest_sync("./docs")
turn = agent.ask_sync("How long are backups retained?")

print(turn.text)
for citation in turn.citations:
    print(citation.title, citation.position, citation.locator)
agent.close_sync()

from_env() is the shortest path. To configure in code instead of by environment — which is what a test or a library consumer wants — build the settings tree and pass it:

from pydantic_rag_agent import RagAgent, Settings

settings = Settings(
    provider={"api_key": "…"},
    vector_store={"backend": "memory"},
    storage={
        "documents": {"backend": "local", "local_path": "./data/documents"},
        "images": {"directory": "./data/images"},
    },
)
agent = RagAgent(settings)

Async is the native interface; the *_sync helpers exist so a script need not become async to ask one question. Both call the same code:

async def answer(agent: RagAgent, question: str) -> str:
    async for update in agent.stream(question):
        ...  # tokens, tool calls, completion
    return (await agent.ask(question)).text

The constructor also takes the seams directly — vector_store=, embeddings=, model=, document_store= — which is how this package's own tests run with no network. See public-api.md.

2. Mounted in FastAPI

The router brings routes; your app keeps its lifespan, its auth and its shutdown.

from contextlib import asynccontextmanager

from fastapi import FastAPI

from pydantic_rag_agent import RagAgent
from pydantic_rag_agent.integrations.fastapi import create_router

agent = RagAgent.from_env()


@asynccontextmanager
async def lifespan(app: FastAPI):
    await agent.check_index()
    try:
        yield
    finally:
        await agent.aclose()


app = FastAPI(lifespan=lifespan)
app.include_router(create_router(agent), prefix="/knowledge")

3. Standalone server and clients

pydantic-rag-agent serve                  # 127.0.0.1:8000
pydantic-rag-agent serve --host 0.0.0.0   # reachable from elsewhere — see the warning below

Two clients speak to it. RagClient is HTTP; InProcessClient runs an RagAgent in your own process behind the same KnowledgeClient protocol, so a Streamlit app or a test can swap one for the other without changing the UI code:

from pydantic_rag_agent.client import RagClient  # HTTP
from pydantic_rag_agent.in_process import InProcessClient  # in-process

with RagClient("http://127.0.0.1:8000") as client:
    for document in client.documents():
        print(document["document_id"], document["source_path"])
    for event in client.stream(conversation_id, "How long are backups retained?"):
        ...

with InProcessClient.from_env() as client:  # no server, same interface
    ...

Both return None from download() for an unknown document, so a stale citation is a normal answer rather than an exception.

4. Embedded Streamlit panel

Requires the ui extra. The host owns the RagAgent and closes it; the panel owns its own session_state, namespaced by key, and makes no module-level Streamlit calls — so importing it cannot disturb your page.

import streamlit as st

from pydantic_rag_agent import RagAgent
from pydantic_rag_agent.ui import rag_chat

agent = RagAgent.from_env()
rag_chat(agent, heading="Ask the handbook")

rag_chat_client(client) renders the same panel over any KnowledgeClient instead of an agent — the shape to use when the UI should not hold a provider credential. See the embedding guide.

5. Any OpenAI-speaking chat UI

Point an existing client at the agent instead of building a frontend:

from pydantic_rag_agent.integrations.fastapi import create_openai_router

app.include_router(create_openai_router(agent, model_id="acme-knowledge"), prefix="/openai")

Point the client at https://your-host/openai/v1. Requests are stateless (the client sends the thread), caller-supplied system messages are ignored, and answers carry a Sources: footer. serve exposes the same route at /v1, so a chat UI can be pointed at a running server with no code at all — --openai-model-id chooses the name the client shows. See ADR 0014.

6. Command line

pydantic-rag-agent configure                         # writes .env
pydantic-rag-agent ingest ./docs                     # index, copying each original
pydantic-rag-agent ask "How long are backups retained?"
pydantic-rag-agent chat                              # a loop that keeps context
pydantic-rag-agent documents                         # the index, with document IDs
pydantic-rag-agent stats                             # chunk count and embedding identity
pydantic-rag-agent delete <id>                       # asks first; -y skips
pydantic-rag-agent dev                               # API + Streamlit UI together

The answer streams and ends with a table of sources. --json on ask prints machine-readable output, and --api-url (or PRAGENT_API_URL) makes any command talk to a running server instead of building an agent locally.

Citations download the original

Every answer carries citations, and each one can fetch the document it names. The same capability is reachable from every shape — this is the part that is easy to miss:

turn = agent.ask_sync("How long are backups retained?")
for citation in turn.citations:
    download = agent.download_sync(citation.document_id)
    if download is not None:  # None means no such document
        path = f"./out/{download.filename}"
        open(path, "wb").write(download.data)
        print(download.media_type)  # e.g. text/markdown

download() returns None for an unknown id rather than raising, because a stale citation is a normal thing to hold. The filename comes from the index, so you get handbook.md, not a hash.

On S3 the originals are proxied by default: the route reads the object and streams it, so a citation link is stable and can be re-fetched. Set PRAGENT_STORAGE__DOCUMENTS__S3_DOWNLOAD_MODE=presign to redirect to a presigned URL instead — faster and off your bandwidth, but the link expires. See ADR 0017.

There is no authentication. serve --host 0.0.0.0 exposes every route, including /documents/{id}/download, to anyone who can reach the port, and a document_id is a content hash rather than a permission. Run it on localhost, or put auth in front of it. Auth is deferred in the backlog with the trigger "deployment beyond localhost".

First run: let the wizard write your .env — provider, models, where documents live, which vector store — so setup is configuration rather than reading a reference table:

pydantic-rag-agent configure

Use your own model provider

OpenRouter is the default, not a requirement. Chat is built against any OpenAI-compatible endpoint, so the provider is configuration:

PRAGENT_PROVIDER__BASE_URL=https://api.openai.com/v1
PRAGENT_PROVIDER__MODEL=gpt-4.1-mini
PRAGENT_PROVIDER__API_KEY=sk-…

The model id is bare — no openai: or openrouter: prefix — and the /v1 belongs in the base URL.

Three things that apply to every provider:

• Embeddings are a separate node. Chat and embeddings can come from different places; point PRAGENT_EMBEDDINGS__BASE_URL / __MODEL / __DIMENSIONS at a provider with an OpenAI-compatible embeddings endpoint if your chat provider has none. Set PRAGENT_EMBEDDINGS__API_KEY explicitly when it differs — otherwise it falls back to the provider key.
• Tool calling is required. The agent retrieves by calling search_knowledge_base. A model or server that ignores tools will answer from general knowledge instead of your documents; a question whose answer is only in the corpus is the quickest test.
• Changing the embedding model or width means reindexing. The width is written into the index metadata and checked at startup, so the guard refuses to mix vectors from different models. DIMENSIONS must also match what the model actually emits — 768 for nomic-embed-text, 1024 is a valid Matryoshka width for text-embedding-3-small.

Images are captioned by PRAGENT_PROVIDER__CAPTION_MODEL (default: the chat model). If you ingest images and the chat model has no vision, set a vision-capable caption model.

Every variable, its default and the programmatic equivalent are in docs/reference/configuration.md.

Getting started from the repository

# Clone and enter the project
git clone git@github.com:hanku4u/pydantic-rag-agent.git
cd pydantic-rag-agent

# Create .venv and install the project plus its dependencies
uv sync

# Provide the one credential: chat, captions, and embeddings all use it
cp .env.example .env
# then edit .env and set OPENROUTER_API_KEY

# Index and ask — the agent runs in this process, no server needed
uv run pydantic-rag-agent ingest ./docs
uv run pydantic-rag-agent ask "How long are backups retained?"

# Optional: the API and web UI together
uv run pydantic-rag-agent dev

The terminal answer streams and ends with a table of the sources it cited. The chat UI from dev is at http://127.0.0.1:8501. --api-url makes any command talk to a running server instead of building the agent locally.

Day-to-day commands

Project layout

pydantic-rag-agent/
├── pyproject.toml            # project metadata, dependencies, extras, CLI entry point
├── uv.lock                   # exact resolved versions — commit this
├── .python-version           # interpreter uv should use
├── AGENTS.md                 # orientation for coding agents
├── docs/                     # design, backlog, decisions — start at docs/README.md
├── scripts/                  # release, docs link check, retrieval evaluation
├── src/
│   └── pydantic_rag_agent/
│       ├── __init__.py       # the public surface, re-exported; no import-time side effects
│       ├── agent.py          # RagAgent — the façade every interface goes through
│       ├── client.py         # KnowledgeClient protocol + RagClient (HTTP)
│       ├── in_process.py     # InProcessClient — the same protocol, no server
│       ├── cli.py            # the command line (`typer` + `rich`)
│       ├── config.py         # the PRAGENT_* settings tree
│       ├── extras.py         # MissingExtraError and the extra-availability check
│       ├── domain/           # models and ports; no I/O
│       ├── application/      # ingestion, embedding, retrieval
│       ├── adapters/         # chroma, document stores, loaders, captioning
│       ├── integrations/     # the FastAPI app and routers (the `api` extra)
│       └── ui/               # the Streamlit panel and app (the `ui` extra)
└── README.md

The src/ layout keeps the package importable from an installed build rather than by accident from the working directory, which is what makes uv run pytest behave the same as a real install.

Configuration

Settings are a tree of six nodes — provider, embeddings, vector store, storage, ingestion, agent — read from PRAGENT_* environment variables via pydantic-settings. Nesting is addressed with a double underscore: PRAGENT_PROVIDER__MODEL, PRAGENT_AGENT__TOP_K. Each node can also be constructed in code, which is how the library is used.

Credentials keep their conventional names (OPENROUTER_API_KEY) and are never read from a committed file. Every variable, its default, and the mapping from the pre-0.1.0 flat names are in docs/reference/configuration.md.

The .gitignore ignores .env and any .env.* file except .env.example.

Contributing

1. uv sync to get a working environment.
2. Pick a feature ID from the backlog and reference it in your commit subject.
3. Add tests under tests/ next to the module they cover.
4. uv run pytest before opening a pull request.

License

MIT. Use it, fork it, ship it — just keep the copyright notice.

The dependency stack is deliberately permissive end to end; the one AGPL package that previously blocked this was replaced. See docs/reference/dependencies.md for the details, and ADR 0007 for the reasoning.