chirppal 0.3.0

One-line, fail-open wrapper for LLM calls — silent-breakage detection for indie AI apps.

chirppal (Python)

One-line, fail-open wrapper for LLM calls — a smoke alarm for the AI features in your app. Wrap a call and ChirpPal watches it for silent breakage (refusals, malformed JSON, empty output, latency spikes) and fire-and-forget reports metadata only to the ChirpPal backend. It never changes what your call returns, raises, or how long it takes.

• Fail-open — any error inside the wrapper (a check bug, a network blip) is swallowed. track()'s return value, exception, and effective timing always match calling your function directly.
• Privacy-first — by default only metadata leaves your process (pass/fail, latency, model, which check failed, and a short error signature rather than the exception's message). Prompt/response content is sent only if you explicitly opt into semantic-judge sampling (an experimental feature, not generally available yet).
• Zero dependencies — standard library only.

Install

pip install chirppal

Quickstart

project= is the one thing you always pass — it selects that project's checks, default model, sampling rate, and API key from chirppal.config.json (see below). Create the config first, then wrap the call:

{
  "config_version": 2,
  "projects": {
    "prod-support-bot": {
      "key_env": "CHIRPPAL_KEY_SUPPORT",
      "checks": { "no_error": true, "not_empty": true }
    }
  }
}

CHIRPPAL_KEY_SUPPORT=cp_live_...     # this project's key, from the dashboard

from chirppal import track

resp = track(
    lambda: client.responses.create(model="gpt-4.1", input=prompt),
    model="gpt-4.1",              # pass the exact model id so deprecation radar can flag it
    project="prod-support-bot",
)

track() returns exactly what the wrapped call returns (and re-raises its exceptions unchanged). No config file at all? track() still works — it just monitors nothing and reports under a dev-only fallback key, exactly like before you had a config.

# Optional: override the endpoint. Defaults to https://api.chirppal.com/api/ingest.
# For local development against your own backend:
CHIRPPAL_ENDPOINT=http://localhost:8000/api/ingest

Config: one project = one policy

A chirppal.config.json at your repo root declares one or more projects — each is a full policy: which checks to run, which environment variable holds its key (key_env), an optional default model, and an optional sampling_rate for passing calls (failures always send regardless). Full format spec, shared with the JS/TS client: CONFIG.md.

{
  "config_version": 2,
  "projects": {
    "support-bot": {
      "key_env": "CHIRPPAL_KEY_SUPPORT",
      "model": "gpt-4.1",
      "sampling_rate": 0.25,
      "checks": {
        "max_latency_ms": 3000,
        "no_refusal": true,
        "valid_json": { "required_keys": ["status"] }
      }
    },
    "billing-bot": {
      "key_env": "CHIRPPAL_KEY_BILLING",
      "checks": { "no_error": true, "must_contain": ["invoice"] }
    }
  }
}

# reads checks + model + sampling_rate + key from the "support-bot" entry above
track(lambda: call_llm(), project="support-bot")

Running two projects (each with its own key) from the same process just means using each project's name in its own track() calls — no extra setup:

track(lambda: call_support_bot(), project="support-bot")
track(lambda: call_billing_bot(), project="billing-bot")

A code-level checks=[...] list still works as an escape hatch — it replaces just the checks for that call; key_env/model/sampling_rate from the named project still apply:

from chirppal import track, checks

track(
    lambda: call_llm(),
    checks=[checks.valid_json(required_keys=["status"]), checks.no_refusal()],
    project="support-bot",
)

model=/sampling_rate= passed to track() likewise override that project's JSON defaults for a single call (e.g. a dynamically-chosen model).

Reading the config is itself fail-open: a missing or broken config, an unset key_env variable, or an unknown project name all warn once and fall back to monitoring less (never checks, or a dev-only fallback key) — never breaks track().

Non-text models (image, audio, embedding, realtime, …)

Wrapping is safe on any model — fail-open means the call's return value, exceptions, and timing are untouched whatever it returns. The model-agnostic signals keep working too: max_latency_ms, no_error, token usage, and the deprecation radar (which just matches the model id you report).

What's text-specific is the content checks — not_empty, valid_json, no_refusal, must_contain, must_not_contain. The built-in adapter only recognizes text/chat responses (a plain string, .text, OpenAI output_text or choices[0].message.content, Anthropic content[0].text). On a shape it doesn't recognize (image, audio, embedding, realtime), those checks skip rather than fail — never a false alarm — and the event's Integration health shows response_shape_recognized: false. To run a content check on a non-text response anyway, pass extract= to turn it into the string you want checked:

track(
    lambda: client.images.generate(model="gpt-image-2", prompt=p),
    project="image-gen",
    extract=lambda r: r.data[0].url,   # now not_empty / must_contain apply to the URL
)

Matching an event back to your own system

Pass metadata= to tag an event with your own identifiers (e.g. a user or request id), and check the dashboard's per-event detail against your own logs:

track(
    lambda: call_llm(),
    project="support-bot",
    metadata={"user_id": "u_123", "api_call_id": req.id},
)

You can also set a static default per project in chirppal.config.json ("metadata": {"env": "production"}) — a per-call value merges with it and wins on a shared key. Bounded to 10 keys / string or number values / 200-char strings; anything past that is dropped or truncated (never an error) and surfaces on the dashboard's Integration health panel. Not encrypted — never put API keys, passwords, or personal data in it.

Every failed check also reports a safe-subset detail (e.g. which keyword was missing, or that JSON parsing failed at a given line/column) — visible on the dashboard's event list — and every event carries a small Integration health snapshot (is the response shape recognized, was anything clamped or truncated, what config is actually in effect) that the dashboard's project page shows as green/yellow. See CONFIG.md and CONTRACT.md §6 for the full details.

Privacy

The default payload is metadata only — no conversation content. prompt and output are sent only on a call you opted into judge-sampling for (judge_sampling_rate > 0 — an experimental feature, not generally available yet). No other path ever sends your users' prompts or your model's output.

error defaults to a short signature (the exception's type, plus a numeric status/error code when the provider exposes one) rather than its message — this is a technical guarantee, not just a convention: the status/code are validated against a strict format before they're included, so a real prompt/response fragment can't slip through as one. Set "send_error_messages": true on a project in chirppal.config.json if you want the full exception message instead (truncated to 500 chars) — only do this if you're confident that message can't echo back your own content.

metadata is a separate, unrelated opt-in (see above) that's sent on every event once you set it — it's not encrypted, so treat it like a log line, not a vault.

Publishing (maintainers)

Requires a PyPI account + API token — this step is done by hand, not by CI:

cd sdk/chirppal
python -m build            # produces dist/*.whl and dist/*.tar.gz
python -m twine upload dist/*

License

MIT — see LICENSE.