Wassitai

Ask an LLM a question, get back a validated Python object — not a string you have to parse.

Wassitai sends your prompt with a schema, picks the best structured-output method the provider supports, then extracts, repairs, validates and coerces the reply through a deterministic pipeline. If the result still doesn't fit the schema, it re-asks the model. Ships as the importable package wassitai, with a REST API and a browser playground on top.

pip install wassitai            # library
pip install "wassitai[api]"     # + REST API and playground

Code written against the pre-0.2 name still works: llm_parser remains importable as a deprecated alias for wassitai until 2.0. If you have an old editable install of the llm_parser distribution, run pip uninstall llm_parser first — pip won't replace it, and its source tree would keep shadowing the alias.

Features

• 🤖 Any OpenAI-compatible provider — Groq, OpenAI, DeepSeek and OpenRouter as presets; Together, xAI, Ollama, vLLM or a local server with just base_url= and model=
• 🧱 Five ways to write a schema — Pydantic model, {"field": "type"} dict, JSON Schema, or a JSON or XML schema string
• 🔁 Object, JSON or XML — in and out — parse either format the model answers in, render the validated result as either
• 📊 Lists — ask for list[Invoice], get a list back
• 🧪 Repairs before it fails — lifts JSON out of markdown fences and prose, fixes trailing commas, smart quotes and Python-repr output, then re-asks the model if it still doesn't fit
• 🔒 Strict or flexible validation — exact types for production, coercion while prototyping
• 💡 Diagnostics you can act on — every stage reports what it did, a confidence score falls as repairs accumulate, and errors are a typed tree with fixes in the message
• 🎯 Pydantic all the way through — your model is the schema, the validator and the return type
• 🌐 A playground, not just a library — browser console plus a beginner's guide
• ✅ 1,100+ tests — unit, integration and end-to-end, no network calls unless you opt in to the live provider test (see below)

Python

from wassitai import Client
from pydantic import BaseModel

class Invoice(BaseModel):
    number: str
    total: float

client = Client.from_provider("groq", api_key="…", model="llama-3.3-70b-versatile")
invoice = client.generate("Extract the invoice from: …", schema=Invoice)   # -> Invoice

# A list, by asking for a list schema:
invoices = client.generate("Extract all invoices: …", schema=list[Invoice])

# Never raises — for batch pipelines:
res = client.try_generate("…", schema=Invoice)     # -> Result[Invoice]
if res.ok:
    print(res.value, res.confidence)

# Text you already have, no provider call:
from wassitai import parse, render
invoice = parse('{"number": "A1", "total": 9.5}', Invoice)
text = render(invoice, "json")

A schema can be a Pydantic model, a plain {"field": "type"} dict, or a JSON Schema. strict=True enforces exact types; the default coerces close-enough values ("3" → 3). max_retries caps how often the model is re-asked (default 1). One dict form is the exception: a dict written as a real JSON Schema (a properties/type object) only checks that required keys are present — declared field types and strict are not enforced, since doing that honestly needs a JSON-Schema-to-Pydantic converter this library doesn't have yet. The {"field": "type"} and {"model": ..., "attributes": {...}} dict forms, and a JSON/XML schema string, are all type-checked like a Pydantic model.

When the model answers in XML

Nothing asks for XML — structured-output APIs speak JSON — but models ignore instructions, and stored answers get replayed through parse(). An XML body is read into the same validated object a JSON one produces:

parse("<person><name>Ada</name><age>36</age></person>", Person)   # Person(name='Ada', age=36)
parse('<person name="Ada" age="36"/>', Person)                    # the same
parse("<ns:person><ns:name>Ada</ns:name></ns:person>", Person)    # and the same

The format is detected from the answer's own shape, so a JSON body is never read as XML (nor the reverse) — including a JSON answer behind a <think>…</think> tag, and an XML answer whose text happens to contain braces.

Four things XML cannot express, and what each one does. Every one of them records a diagnostic, so the intervention is visible in result.diagnostics rather than silent:

strict=True cannot enforce scalar types on an XML body, because the format does not carry them: every leaf xmltodict returns is a string, so <age>36</age> and <age>"36"</age> are the same document. Declared int/float/bool fields are therefore converted before validation in both modes, and the conversion is recorded as a diagnostic. Strict mode still means exactly what it meant for a JSON body — the model sent the wrong type — which is a claim XML has no way to make. A value that is genuinely not a number (<age>twelve</age>) is left alone and still fails.

Capturing failed runs

Turn on capture to write failing and repaired runs to disk, so the pipeline can be improved from evidence rather than memory.

from wassitai import Capture, Client, JsonlSink

with Capture(
    sink=JsonlSink("./captures"),
    experiment="tool-calling-vs-prompt-repair",
) as capture:
    client = Client.from_provider("groq", api_key="…", capture=capture)
    client.try_generate("List the invoice lines", schema=Invoice)

print(capture.stats())

Two files appear in ./captures per process: a .jsonl of records and a .manifest.json holding counters. The manifest's counters.runs counts every run, including the clean ones — the denominator that makes two experiments comparable.

By default, runs that fail and runs that needed repair are captured; pass when="failures" or when="all" to change that. (Pass when="failures_and_repairs" explicitly to document the chosen policy in the manifest.) The repaired outcome covers two different mechanisms — a fix applied inside the pipeline, where the record's result.attempts is 1, and a re-ask of the model, where it is more than 1 — so repaired on its own does not mean "retried"; read attempts to tell the two apart.

What is stored. The prompt, the schema, every attempt's raw answer, the parsed result, the error detail, the provider and model names, strategy, config, token usage, and timestamps. Records are written unredacted, so treat them as sensitive — the parsed result in particular carries the model's structured output. Diagnostic messages are redacted too — they can carry the model's output verbatim, so the redactor is applied to prompts, answers, result, error fields and diagnostic text alike. Three fields are exempt from redaction: request.schema, request.config and metadata are stored unredacted, so a schema supplied as a plain dict — whose values you chose — is written exactly as you passed it. All three are values you supplied rather than anything the model produced, which is why the redactor is not run over them; the API-key scrubber below still is, and so is the max_field_bytes cap — an exemption from redaction is not an exemption from being bounded, and schema is caller-supplied over HTTP. A field shortened that way is named in truncated, as always.

Capture(sink=JsonlSink("./captures"), experiment="e",
        redact=lambda text: EMAIL_RE.sub("[email]", text),
        max_field_bytes=16_384)

Text fields are capped at max_field_bytes (default 65536, None for unlimited), and any field that was shortened is listed in the record's truncated array — a record never silently misrepresents what the model said. No API key is ever taken from the request side: a record is assembled from the schema, the model's answer and the result, and credentials are in scope of none of them. The response side needs more than that, because a provider's error body reaches the record through the exception message (wassitai/providers/transport.py puts up to 300 characters of it into AuthenticationError and RateLimitError), and some providers echo the submitted key in a 401 body. So the key you configured is removed from every string in the finished record, automatically and before your own redact callable ever runs — including the three fields exempt from redaction above. You do not need to write a redactor for it.

If the sink fails — a full disk, a read-only volume — capture warns once, disables itself, and your run continues. A problem with one record is handled differently: a broken redact callable, or a value that cannot be serialised at all, drops that record, counts it in dropped, and warns once, but capture continues. Pass strict_sink=True to make either raise instead.

REST API & playground

uvicorn wassitai.api.app:app --reload

Playground at http://localhost:8000/, beginner's guide at /static/guide.html, OpenAPI reference at /docs.

curl -X POST http://localhost:8000/v1/generate \
  -H "X-Provider-Key: $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"provider":"groq","prompt":"Extract the invoice: …",
       "schema":{"number":"str","total":"float"}}'

Provider keys travel per-request in the X-Provider-Key header (or come from <PROVIDER>_API_KEY on the server) and are never stored, logged, or pre-filled into the browser. Errors map to status codes: 422 validation · 401 auth · 429 rate limit · 502 provider · 504 timeout.

The server bounds request size. A body over WASSITAI_MAX_BODY_BYTES (1 MiB by default) is refused with 413 before it's read into memory; a text field over WASSITAI_MAX_TEXT_CHARS (262144 characters by default) on /v1/parse is refused with 422, naming the field. Set either to 0 to disable it. Both are backstops — the shipped Dockerfile runs uvicorn directly and render.yaml puts nothing in front of it — not a substitute for a limit at your own proxy. /v1/generate's prompt has no character cap of its own — only WASSITAI_MAX_BODY_BYTES bounds it — since it costs provider tokens rather than parse work, and an operator who configures a server-side *_API_KEY has already taken on that spend deliberately.

Capture

/v1/generate and /v1/parse both accept three optional fields:

run_id comes back whenever a run was attempted — success or failure — whether or not include_record was set. It's the handle to quote when asking for a specific run to be deleted, the only workable deletion story for a record that otherwise carries no identity. A request rejected before a run begins has no run to identify, and so no run_id either — no API key at all (401), an unknown provider or a strategy/model combination that can't be built, and a schema that fails to resolve all reject the request before anything is attempted.

A key that was sent and the provider refused is the other side of that line, and answers 401 too: that request was built, dispatched and answered, so it is a run like any other and does come back with a run_id. The distinction is what the id is for — there is a run to point at only in the second case.

curl -X POST http://localhost:8000/v1/parse \
  -H "Content-Type: application/json" \
  -d '{"text":"{\"number\":\"A1\",\"total\":9.5}",
       "schema":{"number":"str","total":"float"},
       "include_record":true}'

The server stores a run only when both capture_consent is true and the WASSITAI_CAPTURE_DIR environment variable is set on the server — either alone stores nothing. Stored records are swept for retention at API startup and are deleted after 30 days.

Whether that configuration actually works is a separate question from whether it's set: a directory can be named and still not be writable by the server process — wrong ownership on a mounted disk, a full disk, a permission revoked while the process is running — in which case capture construction or a later write fails, and the server keeps serving normally without storing anything and without saying so on its own. GET /health reports the real answer as storage_active: true only once a persisting capture has actually opened successfully for this process and hasn't since failed a write; false otherwise. It is never inferred from WASSITAI_CAPTURE_DIR being set alone. The hosted playground's consent checkbox derives its copy from this field (wassitai/api/static/app.js) instead of asserting a fixed promise, so a storage failure degrades to an honest "not storing here" — or, if the check itself couldn't be reached, "couldn't be confirmed" — rather than a notice the deployment can't back up.

Records are grouped by experiment label, each with its own writer, and a bare storage_active aggregates across every label already open in the process — which can disagree with what one specific label will actually experience: an unrelated label's sink failure makes the aggregate say false for a label that is itself fine, and a brand-new label opened after the process has already seen many distinct ones can be silently declined (a fixed per-process cap on distinct labels, sized generously for legitimate use) while the aggregate, having never heard of it, still says true. GET /health?experiment=<label> answers for that specific label instead, and is what the playground actually asks — with its own experiment field, including when it's empty.

On the hosted playground, render.yaml sets WASSITAI_CAPTURE_DIR to a mounted persistent disk, so a consented run survives a redeploy and is purged 30 days later. Self-hosting stores nothing until you set WASSITAI_CAPTURE_DIR yourself — and that path needs to sit on storage that survives a restart, or every record is lost the moment the container does. Most platforms only offer that on a paid tier and typically cap such a volume to a single attached instance, which rules out running more than one replica; on Render specifically, both are confirmed in their disk docs. To have a specific run deleted before its 30 days are up, quote its run_id through Contact below.

include_record returns the record if one was produced. When storage isn't active — no consent, or consent without WASSITAI_CAPTURE_DIR set — that's every outcome, the same as the no-consent case. When storage is active (both conditions above), the server persists under the library's default policy, failures_and_repairs — so a clean run that was consented and stored still comes back with record: null, honestly, because nothing was recorded; run_id still comes back regardless.

What is stored. The prompt, the schema, every attempt's raw answer, the parsed result value, the error detail, the provider and model names, the strategy, the config, diagnostics, token usage and timestamps — the same fields Slice 1 stores locally (see "Capturing failed runs" above). Never the API key, the request's IP address, or its headers.

The API caps text fields at 32 KB — half the library default (65536), because a record now travels over the wire. That cap is per field, not per response: a record with many attempts can still add up to more than 32 KB.

Two shapes a consumer should expect:

• A result.value that exceeded the cap is stored as a truncated string, not an object or a list — check its type before assuming it parses.
• notes names any field that had no native JSON form and was stored as a repr() instead, and any field whose value degraded — for example request.model, when the provider in use exposes no model name.

How it works

generate() → schema adapter → strategy → provider → pipeline → validated object.

• Strategy — resolved per provider in order native (JSON-schema mode) → tool_calling → prompt_repair, which is the universal floor and needs nothing from the provider.
• Pipeline — six ordered stages: normalize → extract → syntax_repair → deserialize → validate → coerce. Each records a diagnostic, and the confidence score falls as repairs and re-asks accumulate.
• Extending — Provider, Strategy, SchemaAdapter and Stage are the four interfaces; most providers need only a preset in wassitai/providers/__init__.py.

Errors form one tree under LLMParserError: ConfigurationError (your bug), ProviderError (the network/provider's), and ParseError (the model's).

Providers

Built-in presets speak the OpenAI wire format: groq, openai, deepseek, openrouter. Any other OpenAI-compatible endpoint (Together, xAI, Ollama, vLLM, local) works by passing base_url= and model=. Native adapters for Anthropic, Gemini and Bedrock are planned behind the same Provider interface.

Why Wassitai

Key differentiators

1. Dual validation modes — coerce close-enough values while prototyping, enforce exact types in production, without changing your schema.
2. It repairs before it gives up — malformed JSON is fixed, and a reply that still doesn't fit the schema is sent back to the model rather than thrown at you.
3. It tells the truth about what happened — every stage records a diagnostic, and confidence drops as repairs and re-asks accumulate, so a "valid" object you shouldn't trust is visible.
4. Any OpenAI-compatible endpoint — Together, xAI, Ollama, vLLM or a local server need a URL and a model name, not an adapter.
5. JSON and XML — an answer in either format parses to the same validated object, which renders back to either.
6. A playground, not just a library — a browser console that exercises the real REST API, with a guide written for people who have never called an LLM.

Use cases

• Development — flexible validation for rapid prototyping
• Production — strict validation for data integrity
• Research — compare providers and strategies on the same prompt and schema
• Education — plain-language guide and honest error messages
• Experimentation — the playground, with no code to write

From a checkout

pip install -e ".[api,dev]"              # editable install with the API and test deps
pytest                                   # 1,100+ tests, no network calls unless you set
                                          # WASSITAI_LIVE and GROQ_API_KEY (opt-in, see tests/bench/)
pytest -m unit                           # or: integration, e2e, provider
pytest --cov=wassitai --cov-report=html

CI runs the same suite on Python 3.9–3.13, plus ruff over the current engine, the REST API and the llm_parser alias (the pre-0.2 core/ and llms/ modules are grandfathered).

Legacy API (removed in 2.0)

LLMParser.ask still works and still supports Gemini and HuggingFace, but is deprecated.

The Streamlit UI at ui/app.py (streamlit run ui/app.py) targets this legacy path; the playground above is the current one.

License

MIT.

Contact

Issues and discussions on GitHub · kaa.kella.abdelaziz@gmail.com