AI Cordon Picket for Haystack

Check what an LLM is given for prompt injection — in both places it can arrive:

The two rule sets are disjoint, and neither is a stricter version of the other — this is not a sensitivity knob. Pick by role: material is what the model works on, the request is what it answers. Your code knows which is which; it puts them in different places when it assembles the call.

The check is a rule, not a model: no GPU, no network, no key, a few hundred kilobytes of base, and a fraction of a millisecond per turn on one core — see what it costs.

Installation

pip install aicordon-haystack

Material at ingest

from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack_integrations.components.preprocessors.aicordon import PromptInjectionFilter

pipe = Pipeline()
pipe.add_component("ipi_filter", PromptInjectionFilter(mode="redact"))
pipe.add_component("splitter", DocumentSplitter(split_by="word", split_length=200))
pipe.add_component("writer", DocumentWriter(document_store=store))

pipe.connect("converter.documents", "ipi_filter.documents")
pipe.connect("ipi_filter.documents", "splitter.documents")
pipe.connect("splitter.documents", "writer.documents")
pipe.connect("ipi_filter.rejected", "quarantine.documents")   # optional; nothing disappears quietly

The component sits before the splitter: a cut here takes the injection out of the chunks, the embeddings and the store at once, with no offsets to reconcile across chunk boundaries.

What it does with a finding

blank is for pipelines that carry offsets, page maps or diffs downstream and cannot have a document change length under them.

The cut takes the line holding the span, or the sentence when that line runs past 1500 characters: the span points at the injection, but what must leave the index is the whole utterance. Measured on 1200 documents: the payload is gone entirely in 91% of catches, at a median 11.6% of the document removed.

The turn the model answers

from haystack_integrations.components.validators.aicordon import PromptInjectionGuard

pipe.add_component("guard", PromptInjectionGuard())          # mode="drop" is the default
pipe.connect("prompt.messages", "guard.messages")
pipe.connect("guard.messages", "llm.messages")               # the model is called on this path
pipe.connect("guard.blocked", "refusal.messages")            # and not on this one

Two sockets, one value. On a flagged exchange run returns blocked and no messages key, so the generator is not called at all. Connect blocked to whatever answers the user instead.

The decision is for the exchange, not for one message: drop the offending turn and the model answers the one before it.

What it reads, and what it does with a finding

roles maps a role to a rule set, default {"user": "dpi"}. assistant is the model's own text, system the operator's. tool carries material and switches on with roles={"user": "dpi", "tool": "ipi"} — measure your own tool output first: over live chat text the ipi rules raise eight times as many alarms as over documents, and they fire on command lists and code, which is what a tool result looks like.

No mode edits a turn, and asking for one raises. The cut above is fitted to an instruction spliced into a document; a typed jailbreak is not spliced into anything — it is the turn, and cutting it leaves the rest of the attack in place.

What lands in the metadata

Written on every document and every message whose role is read, so "checked and clean" is distinguishable from "never checked". A role outside the map gets no fields — a third, distinct fact.

{"ipi_flagged": False, "ipi_action": "none", "ipi_base": "20260817"}
{"ipi_flagged": True,  "ipi_action": "redact", "ipi_base": "20260817",
 "ipi_threats": ["IPI/Secret.Reveal.B"], "ipi_spans": [[812, 947]], "ipi_removed_chars": 163}

{"picket_flagged": True, "picket_action": "drop", "picket_base": "20260817",
 "picket_threats": ["DPI/Policy.Cancel.M"], "picket_spans": [[0, 41]]}

Findings are also logged through Haystack's own logger at warning, so the level is yours to set.

Measured

Not the detector's recall — that ships with the detector — but what the pipeline delivers with the component and without.

Material. Quadrat-IPI v1.0.1, 1000 injected and 1000 clean documents, mode="redact"; how much of a planted payload still reaches the store:

Both columns matter: the first is an arbitrary stream, the second is where the rule is strong. A document cost 8.3 ms in that run — documents are long, and cost follows length.

The request. Held-out forum jailbreaks from TrustAIRLab in-the-wild (537, near-duplicates of the fitting half removed) against 20 000 real user turns from WildChat, mode="drop":

WildChat carries no attack labels and real jailbreaks sit inside it, so "turns not answered" is an upper bound on the cost to a real user, not a false-alarm rate. The detector's working point, on a labelled pool, is in its report.

Reproduce both with eval/measure.py and eval/measure_dialog.py.

What it costs

Adding the component to a pipeline costs 0.39 ms for a turn of median length, and 1.65 ± 0.03 ms averaged over ordinary traffic — 3000 real WildChat turns, each timed five times (eval/costturn.py). Loading the base costs 15 ms, once per process.

The average is four times the median: cost follows turn length, and a chat pool has a long tail. Find your row:

Cost drifts with machine load. These were taken in one run by one procedure — the only way two figures compare.

No findings does not mean no injection.

License

Apache-2.0, the same as the detector it wraps.