stapel-agent

LLM facade: JSON completion, translation, transcription and summarization in front of swappable model / speech-to-text / image-generation providers (Anthropic, OpenAI-compatible, Claude Code CLI, whisper-http, ElevenLabs, AssemblyAI, openai-images), with a prompt cache and a per-call token ledger. Python port of a prior NestJS service — same HTTP paths/contracts, plus an in-process comm surface for monolith deployments.

Part of the Stapel framework — composable Django apps that deploy as a monolith or as microservices without changing module code.

Install

pip install stapel-agent

At a glance

Documentation

capabilities.json · llms.txt (for agents)

What this is

Python port of a prior NestJS service. Same HTTP paths and contracts (stapel-translate's AgentProvider keeps working unchanged), plus a comm surface so monolith deployments call it in-process without HTTP.

Quick start

pip install stapel-agent[anthropic]  # + the Anthropic SDK for the default provider

# settings.py
INSTALLED_APPS = [
    ...
    'stapel_agent',
]

STAPEL_AGENT = {
    "ANTHROPIC_API_KEY": "sk-ant-...",
}

# urls.py — mount the app under agent/
urlpatterns = [
    ...
    path("agent/", include("stapel_agent.urls")),
]

Two surfaces, same contracts:

# HTTP (service-to-service: X-API-KEY, or a staff session)
POST /agent/api/llm/complete   {"prompt": "...", "model": "small|medium|large|xlarge",
                                "provider"?: "...", "system_prompt"?: "...",
                                "images"?: [{"url": "https://..."} | {"data_b64": "...", "mime"?: "image/webp"}]}
POST /agent/api/llm/translate  {"from": "auto", "to": "de", "entries": {"key": "text"}}
POST /agent/api/llm/transcribe {"audio_url": "https://...presigned...", "language"?: "en",
                                "diarization"?: true, "provider"?: "elevenlabs"}
POST /agent/api/llm/summarize  {"text": "..."} | {"transcript": {...llm.transcribe output...}}
POST /agent/api/llm/generate-image {"prompt": "a cat", "size"?: "1024x1024", "n"?: 2}

# comm (in-process in a monolith, transport chosen by STAPEL_COMM)
from stapel_core.comm import call

call("llm.complete", {"prompt": "...", "model": "small"})
call("llm.complete", {"prompt": "what is on this?", "model": "small",
                      "images": [{"url": "https://..."}]})  # url or data_b64 — never raw bytes
call("llm.translate", {"from_lang": "auto", "to": "de", "entries": {...}})
call("llm.transcribe", {"audio_url": "https://..."})  # URLs only — never raw bytes
call("llm.summarize", {"transcript": result["transcript"]})
call("llm.generate_image", {"prompt": "a cat", "n": 2})

LLM failures are HTTP 200 with {"status": "failure", "reason": ...} — 4xx/5xx are reserved for request validation and auth. Successful completions return the parsed JSON in result, prose around it in comment, and snake_case usage (input_tokens / output_tokens).

Every provider call writes a PromptLog row: model, size, source, status, duration, the full token ledger (input / output / thinking / cache-read / cache-write) and what it cost — cost_usd with a cost_basis saying whether that is the provider's own figure, this package's rate card, or unknown. Attribution is the optional user_id / workspace_id pair every llm.* comm function accepts, so per-user and per-tenant cost accounting needs no other table. The cost is stored as computed at call time and never recomputed: re-pricing an old row against today's card would quietly restate history. Transcriptions land there too (source=transcribe, model = STT provider name, LLM token columns NULL but audio_duration_ms carrying the billable audio length and cost_usd the rate card applied to it, the fallback walk in metadata.attempts); each summarize pass is a normal LLM row (source=summarize); image generations log as source=generate_image with {count, mimes, bytes_total} in metadata — image bytes never touch the ledger, and multimodal completions never collide with the text-keyed prompt cache.

Transcription routes through an STT provider chain — explicit provider in the request beats STT_LANGUAGE_ROUTES[lang], which beats DEFAULT_STT_PROVIDER + STT_FALLBACK_CHAIN — and falls back only on transient failures (429/5xx/timeouts); bad audio or auth errors never retry on another provider.

Settings — STAPEL_AGENT

Provider matrix

No CLI in any default path. claude-code is strictly opt-in: it exists for hosts that ship the Claude Code CLI in their image and want the CLI to handle its own authentication (provider: "claude-code" per request, or DEFAULT_PROVIDER override). There is no OAuth credential reading and no background token-refresh — that plumbing was deliberately dropped.

Adding, overriding and removing providers (merge semantics)

STAPEL_AGENT["PROVIDERS"] is an overlay merged over the built-ins, not a replacement dict — adding your provider never requires restating the three shipped ones, and setting a name to None removes it:

# settings.py — one line per change, built-ins stay available
STAPEL_AGENT = {
    "PROVIDERS": {
        "acme": "myproject.llm.AcmeProvider",  # add a custom backend
        "claude-code": None,                    # remove a built-in
    },
    "DEFAULT_PROVIDER": "acme",
}

Or register at runtime from your app's AppConfig.ready() (highest precedence):

from stapel_agent import register_provider

register_provider("acme", AcmeProvider)  # class or dotted path

A custom backend is just a stapel_agent.LlmProvider subclass returning a ProviderResult. Django system checks (stapel_agent.E001/W001/W002) flag a DEFAULT_PROVIDER that is not in the effective registry, unimportable dotted paths and non-LlmProvider entries at startup. See MODULE.md for the full extension-point map.

STT provider matrix

The STT registry has the same merge semantics as the LLM one (STAPEL_AGENT["STT_PROVIDERS"] overlay, None removes, or register_stt_provider() at runtime), and its own startup checks (stapel_agent.W003/W004). A custom engine is a stapel_agent.SttProvider subclass returning a NormalizedTranscript — MODULE.md walks through a self-hosted GigaAM adapter as the worked example.

Vision & image generation

The two vision-capable LLM providers map images into their dialects automatically — anthropic to image content blocks (URL or base64 source), openai-compat to image_url parts (URL or data URI). claude-code has no vision: an image request through it fails fast with status: "failure".

Image generation ships one built-in backend, openai-images — anything speaking the OpenAI POST {base}/images/generations dialect (OpenAI, Together, self-hosted). It is the third instance of the merge-registry pattern (STAPEL_AGENT["IMAGE_PROVIDERS"] overlay, None removes, register_image_provider() at runtime; startup checks stapel_agent.W005/W006). Vendors with their own protocols (Stability, Ideogram, ...) are an app-layer stapel_agent.ImageGenProvider subclass — recipe in MODULE.md. The agent returns raw results and writes the ledger; storage/placement belongs to the calling tier.

Staged analysis (stapel_agent.analysis)

A recognition chain that takes tens of seconds is a JOB, not a request. analysis is that job: one JSON state document per subject, a stage apiece, a fast stage whose answer is readable while a slow one still runs, and a slow stage that writes progress and a growing partial result after every batch rather than only at the end.

from stapel_agent.analysis import (
    Fingerprint, ModelStateStore, ON_PHOTOS, Stage, StageBatch, runner,
)

stages = [
    Stage("text", describe, depends_on=frozenset({ON_PHOTOS})),
    Stage("features", fill),                       # yields StageBatch(...)
    Stage("moderation", screen, blocking=False),   # cannot fail the job
]
store = ModelStateStore()
state, started = runner.start(
    store=store, key=key,
    fingerprint=Fingerprint.of(photos=blobs, text=typed),
    stages=stages, seller_filled=already_answered,
)

The fingerprint has two halves because they invalidate different work: a stage declares which halves it depends on, so a refresh with edited text re-runs the text-dependent stages and keeps the photo stage's answer. start() is idempotent — the same photos and the same text are the same question. AnalysisJobView is the POST/GET surface (?wait=1 waits for the fast stage only), and analysis.blocks orders a catalogue's feature definitions into the order a composer asks them while resolving them in dependency order, giving every un-askable field an explicit reason instead of dropping it.

Privacy & retention

PromptLog is both an accounting ledger and a copy of the conversation, so it is treated as customer content:

• The cache is per-tenant. A cached answer belongs to the user_id that paid for it; a call with no scope skips the cache entirely unless its source is listed in CACHE_ALLOW_UNSCOPED.
• Subject requests reach it. AgentGDPRProvider (section agent) is registered into stapel_core.gdpr.gdpr_registry at app startup — export returns the user's rows, erasure scrubs their text and unlinks the subject while the token counters survive for accounting.
• Text expires. Schedule manage.py purge_prompt_logs (--days N, --dry-run): it scrubs prompt, system prompt, response and error text on rows older than PROMPT_LOG_RETENTION_DAYS and keeps the counters.

License

MIT — see LICENSE.

_{This page is assembled by stapel-readme from docs/readme.md plus the contract artifacts in docs/. Edit the prose in docs/readme.md; the badges, facts and links above and below it are generated — do not hand-edit README.md.}