pantheon-guardrails

Three guardrails for LLM agents, extracted from a private production system:

• ConstitutionScorer — scores a draft reply against a weighted rubric, but only when the draft looks like it's worth paying for.
• SafetyLayer — redacts emails, phone numbers and card-like digit runs, while allowing a business to publish its own contact details.
• CrisisProtocol — detects a user in distress and breaks the assistant's persona.

Apache-2.0. Python 3.11+. One runtime dependency (pydantic, and only for the rubric model).

pip install pantheon-guardrails

What is actually interesting here

Most of this is unremarkable. Two ideas are worth the read.

1. Conditional judging

Using an LLM to grade another LLM's output is a well-known technique and a well-known cost — you double your inference bill to check work that is usually fine.

So the judge doesn't always run. A cheap regex pre-pass decides whether a draft is worth checking, and it fires on the things a customer acts on: money, quoted times and dates, percentages, bookings, refunds, deletions, anything medical or legal.

scorer = ConstitutionScorer(rubric, judge=my_judge)

await scorer.score_if_needed("The garden is lovely this time of year.", manifest)
# -> None. No LLM call. Nothing here can hurt anybody.

await scorer.score_if_needed("I've booked you in for 3pm Tuesday, that's £40.", manifest)
# -> Score(total=0.87, guidance=None). A time, a date and a price: worth checking.

The heuristic is a first cut and is stated as such in the source. It has not been calibrated against a labelled set in this repo. Override _should_judge or pass always_judge=True if your risk profile differs.

2. The judge must be a different model

scorer = ConstitutionScorer(rubric, judge=judge)   # required, not optional

A generator asked to mark its own work shares its own blind spots — the same training, the same failure modes, the same confident wrongness. The judge is a required argument specifically so that "which model grades this?" is a decision you make rather than a default you inherit.

The Judge protocol is two methods wide, so adapting whatever client you already use takes about five lines:

class AnthropicJudge:
    def __init__(self, client, model="claude-haiku-4-5"):
        self.client, self.model = client, model

    async def complete(self, messages):
        r = await self.client.messages.create(
            model=self.model, max_tokens=512,
            system=next(m.content for m in messages if m.role == "system"),
            messages=[{"role": "user", "content": m.content}
                      for m in messages if m.role == "user"],
        )
        return type("C", (), {"text": r.content[0].text})()

The redaction bug worth knowing about

A PII redactor that blanks out phone numbers will, by default, blank out the business's own phone number — turning "call us on 01234 567890" into "call us on [redacted:phone]". The assistant becomes useless at the thing it is asked most often.

layer = SafetyLayer(allow=["01234 567890", "hello@theshop.co.uk"])
layer.screen("Call us on 01234 567890, not 07700 900999").text
# 'Call us on 01234 567890, not [redacted:phone]'

Published contact details survive; a stranger's number does not. This is obvious in hindsight and was not obvious in advance.

Crisis detection: the default is the feature

CrisisProtocol fires on distress and breaks persona. The part that took the work was not firing — "I need to dye my hair" must not trigger a safeguarding response. A false positive here derails an ordinary conversation, which is its own kind of harm.

In the system this came from, the protocol is on by default and must be explicitly opted out of, rather than opted into. That ordering is the actual safety property.

What this is not

Stated plainly, because the alternative is letting you find out yourself:

• Not a complete safety system. These are three narrow controls. Prompt injection, jailbreaks, training-data leakage and tool-use authorisation are all out of scope.
• Not battle-tested at scale. It runs in one production system with a small user base. It has not been attacked by anyone competent.
• Not novel. LLM-as-judge, PII regexes and crisis keyword detection are all prior art. The contributions here are the conditional trigger and the decorrelation requirement, both of which are engineering judgement rather than research.
• The regexes are first cuts. PII detection by regex is inherently incomplete. The high-stakes trigger is English-only and will miss idioms it was not written for.

Provenance

Extracted from PANTHEON, a private multi-tenant agent substrate, in September 2026. The git history is preserved from the original commits — the first is phase 00: bootstrap, 5 June 2026. The extraction removed a dependency on the product's global settings object; the judge is now injected instead of built from a configuration singleton.

The tests were rewritten during extraction. The originals exercised these classes through the agent loop, which meant they tested the loop as much as the guardrail — if a behaviour can only be demonstrated by standing up a whole runtime, it isn't really a library.

Development

uv venv && uv pip install -e ".[dev]"
python -m pytest