PrivAiTe

Self-hosted PII redaction proxy for LLM APIs.

A drop-in LLM proxy that replaces PII before it reaches the provider, including inside tool-call arguments and multimodal content, with zero telemetry.

Told in writing to report its config variables but never their values, Claude Code sent 3 of 4 secrets to its provider anyway: the same secrets also sat in a log file the task had it read. Over that session 23 of 24 planted values reached the provider; through PrivAiTe's agent gateway, 2 of 24. Wire-level captures of real agent sessions, and the two that still get through are documented rather than rounded away: the measurement, what it misses.

Unreleased source update: local rules now target the credential fields behind those historical log misses, and overlapping detections respect irreversible and block policies. These changes are not in the published 0.4.3 package. See formats and limits. In the offline regression replay, 9 of 10 credential occurrences survived in a 69 KB log before the change; none survived afterward. Processing still takes about 25 seconds with onnx.

You type: "Je m'appelle Marie Dupont, email marie@acme.com"
LLM sees: "Je m'appelle <PERSON_1>, email <EMAIL_ADDRESS_1>"
LLM says: "Bonjour <PERSON_1>, votre email <EMAIL_ADDRESS_1> est noté."
You  see: "Bonjour Marie Dupont, votre email marie@acme.com est noté."

PrivAiTe sits between your app and the model provider. It finds names, emails, phones, cards, IBANs, secrets and more, swaps them for stand-ins before anything leaves your machine, and puts the real values back in the reply. Two types are deliberately not put back: both shipped configs mask CREDIT_CARD and redact SECRET (entity overrides), which throws the original away on purpose. Most tools scan only the plain message text; agent traffic hides PII inside tool-call JSON, and that is the gap PrivAiTe closes. Detection runs locally (two engines, Presidio + OpenAI's open privacy-filter model), and the engine runs three ways: standalone proxy, Open WebUI filter, or LiteLLM guardrail.

This is local pseudonymization, not anonymization, and detection is best-effort rather than a guarantee. You remain the data controller. The Threat model spells out exactly what it protects against and what it does not.

Quick start

Docker (fastest): the detection model is baked in, so it runs offline from the first request.

docker run -d -p 8400:8400 \
  -e PRIVAITE_API_KEYS=change-me \
  -e OPENAI_API_KEY=sk-... \
  ghcr.io/crp4222/privaite:0.4.3

The same image is on Docker Hub too: swap the last line for crp4222/privaite:0.4.3 if you prefer pulling from there. Release details: PrivAiTe 0.4.3.

Two keys, two roles: PRIVAITE_API_KEYS is the key your client sends to PrivAiTe (pick any value); OPENAI_API_KEY is your real provider key, which stays in the container and never reaches your client. This exposes gpt-4o-mini and gpt-4o; for any other provider (Ollama, Azure, anything LiteLLM supports), mount a config: configuration.

pip:

python -m pip install --upgrade "privaite>=0.4.3"
# One spaCy model per scanned language; the default preset scans EN + FR.
python -m spacy download en_core_web_lg && python -m spacy download fr_core_news_md

cat > privaite.yaml <<'EOF'
providers:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: ${OPENAI_API_KEY}
pii:
  enabled: true
  preset: onnx    # or "light": faster, no model download, classic PII only
EOF

# Your real provider key: the config above interpolates it, and startup fails if it is unset.
export OPENAI_API_KEY=sk-...

PRIVAITE_API_KEYS=change-me python -m privaite --config privaite.yaml

Connect: point any OpenAI-compatible client at http://localhost:8400/v1 with the key change-me. For Open WebUI: Admin → Settings → Connections → OpenAI API, URL http://localhost:8400/v1 (or http://host.docker.internal:8400/v1 if Open WebUI runs in Docker), key = your PRIVAITE_API_KEYS value. Client snippets (curl, Python, Node) are in examples/. Prefer no separate proxy? Use the in-process Open WebUI filter.

Use it with Claude Code (agent CLI gateway)

Opt-in gateway mode (off by default): your agent CLI points its base URL at PrivAiTe, which scrubs PII and secrets out of each request (tool-call arguments included) with the same local, benchmarked detection, relays whatever auth the CLI itself sends verbatim upstream, and restores the real values in the response, streaming included. The Claude Code path (Anthropic Messages API) is validated live end to end: running against the real Anthropic API with restore disabled proved the provider only ever received placeholders. Codex support is beta (see below). Any OpenAI-compatible app already works through the standard proxy above; the gateway adds the native protocols these CLIs speak.

flowchart LR
    CLI["Agent CLI"] -- "request + the CLI's own auth token" --> PVT["PrivAiTe"]
    PVT -- "placeholders only, token relayed verbatim" --> API["Provider API"]
    API -- "response" --> PVT
    PVT -- "real values restored, streaming included" --> CLI

gateway:
  enabled: true
  anthropic:
    base_url: "https://api.anthropic.com/v1"
pii:
  detection_cache:
    enabled: true    # recommended for agent sessions, see below

ANTHROPIC_BASE_URL=http://localhost:8400 claude

Enable the detection cache when you use the gateway: agent CLIs resend the whole conversation every turn, and without the cache every turn re-scans the entire history (on a large measured session the per-request scrub peaked at 42 s with Claude Code and 72 s with Codex, against a 1 to 3 s median with the cache on). The threat model spells out the memory tradeoff.

Codex (beta). The gateway also exposes /v1/responses (OpenAI Responses API), which is what Codex speaks. That path passes the same test suite but has had less live validation than Claude Code, so it is labeled beta; setup is in docs/gateway.md.

Four things to know before relying on it:

• Measured, not promised. In the historical live agent-workflow benchmark, Claude Code reading a repository with 24 planted PII values and secrets sent all 24 to the provider directly; through the gateway, 0 reached it on the small fixture and 2 of 24 still got through on a larger session. The unreleased source targets those two log-field formats. This does not establish zero leaks on arbitrary agent traffic.
• It protects the egress, not the agent. Claude Code and Codex still hold the real values in their own context and local transcripts; only what reaches the provider is scrubbed.
• The agent's own prompt is deliberately not scanned. The Anthropic system field and the Responses instructions field pass through as-is, and Claude Code injects your CLAUDE.md and project context there.
• Auth is relayed, not managed, so the gateway routes are open. PrivAiTe injects and validates nothing there: with gateway mode on, /v1/messages and /v1/responses accept a request that carries no PRIVAITE_API_KEYS value at all, by design, since the only credential in play is the one your CLI sends upstream. The server also binds 0.0.0.0 by default and applies no rate limit, so an exposed port plus gateway mode is an endpoint anyone who can reach it can drive (on your provider account). Bind it to localhost or keep the port off untrusted networks. Whether your provider's terms of service permit that traffic to transit a local proxy is between you and the provider; this is not a provider-supported integration, and API-key mode is the durable path.

Full setup, the flow diagram in detail, scanned surface and limits: docs/gateway.md.

Benchmark

Measured on 120 real documents from the open AI4Privacy pii-masking-200k dataset (458 PII items, labeled by 10 independent auditor agents and cross-checked against the dataset's own mask) across DE, EN, FR, IT, plus 14 clean documents for false positives.

Read the 100% precisely, it is structural, not absolute: of the PII PrivAiTe detects in plain text, 100% is also removed from tool-call JSON. End to end, its tool-call leak equals its detection misses (15.1% on this corpus with the onnx preset), the same misses flat text has.

Two honesty notes, both favoring caution. LLM Guard's detection model is fine-tuned on the exact dataset behind this corpus, so its recall here is optimistic; PrivAiTe's default model is not (OpenAI's model card states it did not train on it). An out-of-distribution cross-check on two independent corpora confirms the default generalizes: ~84% held on Gretel finance text while the AI4Privacy-tuned model drops to ~62% (OOD_COMPARISON.md).

Rechecked against the unreleased structured-secret source on 2026-09-13: onnx recall and clean-document false positives are unchanged; light span recall rises from 62.4% to 62.7%. The latencies below are means per corpus document from that local run, not large agent-request latency guarantees.

Per-language and per-entity tables, competitor configs, methodology, reproduction: privaite-bench. Feature comparison: docs/comparison.md.

Protocol traces are harder. A separate Privy and Kiji evaluation tests the unreleased source on 300 synthetic JSON, HTML, XML and SQL traces. The current onnx stack fully covers 258/491 annotated spans (52.55%); this stricter character-coverage metric differs from the literal recall above. Replacing Privacy Filter with the tested Kiji ONNX artifact is faster but lowers coverage to 147/491 (29.94%), including only 2/15 password spans versus 12/15. Kiji remains a benchmark experiment, with no new production preset.

The historical live agent-workflow benchmark uses 24 planted values in a repository, real Claude Code and Codex sessions, and a recording proxy. Directly, Claude Code sent 24/24 values and Codex 20/24; through PrivAiTe, none reached the provider on the small fixture and two secrets survived on the larger session. The write-up retains those original results. The new structured-secret rules address the reproduced formats; offline regression replays do not replace the live-session measurements.

Presets

*Span recall on the AI4Privacy benchmark above. max adds GLiNER (trained on data independent of AI4Privacy): on out-of-distribution corpora it raises recall by several points at the cost of more false positives and a torch dependency (pip install 'privaite[gliner]'); with it selected but not installed, the proxy fails at startup with an install hint rather than silently degrading.

onnx combines contextual recognition with structured rules. light uses Presidio and, in the unreleased source, the same structured-secret rules; it has no contextual Privacy Filter model. How the engines work and what stays off by default: docs/detection.md.

Footgun: do not pin detectors.presidio.entities to a short allowlist on the light path. It restricts detection to only those types and roughly halves recall (to ~36%). Leave entities unset; the proxy logs a warning at startup if it detects a low-recall configuration.

Your policy, your types

The presets are the statistical half of the answer: a benchmarked detector with a measured recall. The other half is declarative, and it is yours. What counts as sensitive in your deployment is written in YAML, no retraining involved: define your own types with regex custom_patterns, decide each type's fate with entity_overrides (restored, faked, masked, or destroyed), and list what must never leave at all, even as a placeholder, under block_entities. The whole policy is deterministic, can be dry-run before you trust it, and the proxy refuses to start if a block rule can never fire. The narrative and a worked example: docs/policy.md.

What it scans

Before anything is forwarded: messages[].content (plain string or multimodal text parts), tool_calls[].function.arguments and the legacy function_call.arguments (parsed as JSON, scrubbed value by value including bare numeric leaves, keys and function names intact), /v1/completions prompt and suffix, /v1/embeddings input, chat prediction.content (predicted outputs) and web_search_options.user_location. On the way back, values are restored in content, tool calls, reasoning traces, refusals and audio transcripts, streaming included.

NOT scanned (know your surface): messages[].name, top-level user/metadata, tools definitions, JSON object keys, and tokenized (integer-array) inputs, which carry no text to inspect. Keep PII out of those fields. Endpoints, strict mode and passthrough caveats: docs/api.md.

Threat model

PrivAiTe performs local pseudonymization, not guaranteed anonymization. Detection runs on your machine; the real ↔ placeholder mapping lives in memory only for the duration of a request and is dropped afterwards.

Identical ONNX windows can reuse detection predictions within that same scrub operation. This bounded cache contains salted input hashes and detection labels/ scores only, uses the current text's offsets, and is cleared on completion or cancellation. It does not retain input text, token IDs or mappings between requests. See request-local window reuse.

What it protects against: the LLM provider storing, training on, or logging your raw PII. The provider receives placeholders (<PERSON_1>, …) for everything the detector catches, across message content, tool-call arguments, and multimodal text.

What it does NOT protect against:

• PII the detector misses. Detection is statistical and never 100% (see the benchmark). A name it doesn't recognize reaches the provider. Treat the output as best-effort, not a guarantee.
• Unrecognized secret formats. Context changed the model's predictions in the historical log benchmark. The unreleased source adds rules for common credential assignments, URI passwords and bearer headers, including that fixture's field names. Unknown names, encoded or split values, and bare values without their field context can still survive. This affects every surface that uses the engine. Supported formats and remaining boundary limits are in detection.
• Re-identification from context. Even with names replaced, the surrounding text can stay identifying ("the CEO of <ORG_1> who resigned in March").
• A compromised local machine. The mapping and raw text live in local memory; this is not a defense against a local attacker.
• The provider correlating requests within a session.
• A model inventing replacement values. Restoration requires the model to preserve the placeholder. The English agent instructions help a cooperative model copy placeholders, including in tool arguments; they cannot enforce its behavior or repair a missed detection.
• The agent itself, in gateway mode. The CLI keeps the real values in its own context and local transcripts; only the traffic to the provider is scrubbed. And the agent's own prompt (the Anthropic system field, the Responses instructions field) is relayed unscanned, so PII in your CLAUDE.md or injected project context reaches the provider.

If you enable the detection cache (pii.detection_cache, off by default), one nuance is added to the promise above. The reversible mapping is still per-request and still dropped when the request ends. But the cache keeps PII-derived metadata in process memory for up to its TTL (default 30 minutes) after a request ends: salted BLAKE2b hashes of recently scanned text fragments, plus the positions, types, scores and detector sources of the PII spans found in them. An expired entry is never served again; it is removed from memory on the first cache write after its expiry, and the whole cache is cleared at shutdown, so only a process that goes completely idle keeps its last (expired, unusable) entries longer, until that next write or shutdown. No text, no PII values, no anonymized output, and nothing on disk. The honest delta: an attacker who can already read process memory (who today sees every in-flight request and its full mapping) additionally gains, for up to the TTL after traffic stops (longer only in the idle-process case above), (a) confirmation that a specific candidate text was recently processed, since the hash salt sits in the same memory, and (b) the positions and types of PII inside documents they obtained elsewhere. They gain no raw values and no ability to reverse placeholders. In multi-user deployments there is also a dedup timing side channel: the cache is shared across auth keys, and a cache hit is observably faster than a miss, so one user can in principle probe whether an exact text was recently sent by another. Leave the cache off if any of this matters for your deployment; enable it for agent CLI sessions, where it removes the cost of re-scanning the entire resent conversation on every turn.

For GDPR/HIPAA: treat this as pseudonymization + transfer minimization, not anonymization. If you need irreversible removal, use method: "redact"; the shipped configs already do that for SECRET and mask CREDIT_CARD, per-type, on top of reversible placeholders for everything else (entity overrides). Audit it on your own data: docs/verify.md.

Alternatives

Keeping PII out of LLM calls is a crowded space, and PrivAiTe is not always the right pick. Based on each project's public docs as of June 2026:

• LiteLLM has a built-in Presidio guardrail, the natural choice if you already run the LiteLLM proxy and want PII handling inline (there are a few open bugs around scrubbing requests and responses).
• Managed/cloud options exist too, such as Microsoft PII Shield and LangChain's gateway redaction.

Where PrivAiTe differs: it anonymizes PII inside tool-call arguments and multimodal content, not just message text (LangChain's gateway docs, for instance, note that tool-call arguments are not scanned), it restores the original values in the response, and it ships a reproducible benchmark. If your traffic is agentic or multimodal, that gap is the reason this exists.

Integrations

• Open WebUI filter (setup, hub listing): an Open WebUI Filter Function running the engine in-process, no separate proxy. Admin Panel → Functions → paste integrations/openwebui/privaite_filter.py, enable, pick preset and languages in its valves. Covers message text, tool calls and multimodal.
• LiteLLM guardrail (setup): a custom guardrail for teams already on the LiteLLM proxy. Mount integrations/litellm/privaite_guardrail.py next to your config.yaml to anonymize requests and restore responses inline, including tool-call arguments, which LiteLLM's built-in Presidio guardrail does not scan.

Docs

Also browsable as a site: crp4222.github.io/PrivAiTe.

• How detection works: the two engines, what each catches, what stays off by default, known limitations
• Configuration reference: providers, Docker with custom config, anonymization methods, block_entities, custom patterns, languages, pinned model revisions
• Your policy, your types: the declarative policy layer as one story: custom types, per-type fates, hard blocks, dry-run, and where its determinism ends
• API reference: endpoints, the exact scanned/unscanned surface, strict mode, passthrough caveats
• Verify what gets redacted: audit the proxy on your own data, dry-run inspect endpoint
• Feature comparison and the reproducible benchmark
• Agent CLI gateway: Claude Code setup, Codex setup (beta), what gateway routes scan, honest limits
• Changelog

Development

git clone https://github.com/crp4222/PrivAiTe && cd PrivAiTe
pip install -e ".[dev]"
python -m spacy download en_core_web_lg && python -m spacy download fr_core_news_md

cp .env.example .env                                    # keys
cp config/privaite.example.yaml config/privaite.yaml    # providers
python -m privaite --reload                             # dev mode (auto-reload)

python -m pytest tests/ -v

License

BSD 3-Clause. See LICENSE.