pandion-api

pandion-api serves Pandion over REST. It is a FastAPI app factory: create_app gives you a FastAPI app with Pandion's documents, runs, questions, evaluations and comparisons as HTTP routes. RAG Playground mounts it at /api/v2.

Every route that takes a pipeline takes the pipeline file's shape as JSON, the same dict Pipeline.from_dict reads. Everything a site wants to limit is one Policy object, and the default allows everything.

Install

pip install pandion-api

It needs Python 3.13 and works with the Pandion versions its pyproject.toml declares (pandion>=0.8.0,<0.9 today), which pip installs for you. The pipeline's store needs its extra, for example pip install pandion-api "pandion[lancedb]".

docs/pandion-api/index.md is the full reference. Changelog: what changed in each release.

Run it

# serve.py
from pathlib import Path

from pandion_api import ApiSettings, create_app

app = create_app(ApiSettings(data_dir=Path("./pandion-data"), prefix="/api/v2"))

uvicorn serve:app

With no prefix the routes sit at the root, so once it runs, curl http://127.0.0.1:8000/health (or /api/v2/health with the prefix above) is the first call. The reference has a step by step first run.

ApiSettings takes data_dir, the one folder every request runs under. It also takes models (a pandion.ModelSettings), prefix (where the routes are mounted) and title. Pass a Policy as the second argument to create_app to set your site's rules.

examples/pandion-api/ has a server script, a Python client that validates, uploads, indexes, asks and evaluates, the same steps as curl commands, and a small Policy.

Serve mode

For a site's own users, create_app(settings, policy, mode="serve", pipeline="pipeline.yaml") loads one pipeline file once and answers POST /chat, as JSON or as server-sent events. Your Policy says who is asking (user_id), which roles they have (roles) and whether they may ask now (limit). Serve refuses to start without a Policy that overrides all three; pass Policy.anonymous() to say on purpose that everyone is one user. Build the index first with pandion sync.

Serve is hardened by default: generic errors with a trace id, model keys from the environment only, public model endpoints only, a limit of 20 turns a minute and 2 active turns for each user, a 64 KB request body, a 120 second turn, no /docs and no CORS. See Serve mode: hardened defaults. serve_policy.py shows a Policy that checks a signed gateway token.

curl -N http://127.0.0.1:8000/chat -H 'content-type: application/json' \
  -d '{"messages": [{"role": "user", "content": "How long is the notice period?"}], "stream": true}'

Serve also has POST /ask (one question, nothing stored), GET /conversations, GET and DELETE /conversations/{id}, GET /memories and DELETE /memories/{id}, and POST /feedback (up or down on a trace). A user only ever sees their own; anything else is a 404.

docs/pandion-api/serve-mode.md has the request, the events and /ready.

Serve deletes conversations older than ServeSettings(retention_days=90) every day, and pandion forget-user removes one user's data. See Your users' data.

Serve mode logs one JSON line per request, step and failure, and sets an X-Trace-Id header. For OpenTelemetry spans, install pandion-api[otel] and set OTEL_EXPORTER_OTLP_ENDPOINT. See Logs and traces. For Prometheus metrics on their own port, install pandion-api[metrics] and set ServeSettings(metrics_port=9464). See Metrics.

The routes

Each path is under prefix.

Every run's result carries usage on each step (the token counts and model ids of its model call, or null) and cost on each report (seconds, tokens and estimated dollars). Speed and cost lists the fields.

The Policy

A site subclasses pandion_api.Policy and overrides only what it needs. A method refuses by raising fastapi.HTTPException. These are the hooks, from pandion_api/policy.py:

• dependencies(): FastAPI dependencies to run on every route, such as one that sets a cookie.
• can_read(sha, request): may this request read this document?
• check_upload(file, request): read an upload and return its bytes, or refuse it.
• check_document(data, filename, request): refuse a document's bytes, for an upload and a URL document alike.
• check_fetch(url, request): refuse a link before it is fetched.
• fetch_slot(request): a context manager the fetch runs inside, for example to limit fetches at once.
• max_url_bytes(request): the largest page a link may fetch (20 MB by default).
• on_uploaded(document, request): called right after an upload's record is written.
• on_delete(sha, document, request): refuse a delete, or say whether to delete the file now.
• describe(document, request): extra keys to add to a document in the list.
• private(sha, request): is this document one caller's own, so its page images are cached privately?
• keeps_question_sets(request): keep a checked question set, or only check it?
• max_active_runs(): how many runs may work at once.
• strategies(catalog, request): the strategy list a caller is shown.
• check_run(config, request): refuse a pipeline before it runs.
• public_endpoints_only(request): accept only a public https address for a custom model endpoint?
• max_variants(request): how many variants one comparison may run.
• check_question_count(kind, count, request): refuse an evaluation or comparison with too many questions.
• max_runs_per_owner(request): how many runs one owner may have at once.
• run_owner(request): who a run belongs to.
• check_clear_cache(request): refuse DELETE /cache.
• server_key(provider): the key the server itself has for a provider, and where it came from.
• key(provider, request): the key for this request, and where it came from.
• credentials(request): the keys a run gets.
• missing_key(provider): what a key check says when there is no key.
• check_provider(provider, request): refuse a key check for a provider.
• scrub(text): an error message as it may leave the server.
• on_startup(app) and on_shutdown(app): called once as the server starts and stops.

A complete, tested example is policy_example.py: a SizeLimitedPolicy that changes only check_document to refuse a document over a size limit with a 413.

Two class attributes set wording: not_readable is the message for a document the caller may not read, and not_keeping_note is what a question set report says when the set was not kept.

How the playground uses it

The playground's backend, in playground/backend/, calls create_app with ApiSettings(prefix="/api/v2") and PlaygroundPolicy, the hosted demo's rules. It then adds its own routes under /api and serves the web app.