privaite 0.3.1

Drop-in self-hosted LLM proxy that reversibly redacts PII before LLM calls, including tool-call arguments and multimodal content

PrivAiTe

Self-hosted PII redaction proxy for LLM APIs.

A drop-in LLM proxy that reversibly replaces PII before it reaches the provider, including inside tool-call arguments and multimodal content, with zero telemetry.

You type: "Je m'appelle Marie Dupont, email [e-mail]"
LLM sees: "Je m'appelle <PERSON_1>, email <EMAIL_ADDRESS_1>"
LLM says: "Bonjour <PERSON_1>, votre email <EMAIL_ADDRESS_1> est noté."
You  see: "Bonjour Marie Dupont, votre email [e-mail] est noté."

PrivAiTe sits between your app and the model provider. It finds names, emails, phones, cards, IBANs, secrets and more, swaps them for stand-ins before anything leaves your machine, and puts the real values back in the reply. Most tools scan only the plain message text; agent traffic hides PII inside tool-call JSON, and that is the gap PrivAiTe closes. Detection runs locally (two engines, Presidio + OpenAI's open privacy-filter model), nothing phones home, and the same engine runs three ways: standalone proxy, Open WebUI filter, or LiteLLM guardrail.

This is local pseudonymization, not anonymization, and detection is best-effort rather than a guarantee. You remain the data controller. The Threat model spells out exactly what it protects against and what it does not.

Quick start

Docker (fastest): the detection model is baked in, so it runs offline from the first request.

docker run -d -p 8400:8400 \
  -e PRIVAITE_API_KEYS=change-me \
  -e OPENAI_API_KEY=sk-... \
  ghcr.io/crp4222/privaite

Two keys, two roles: PRIVAITE_API_KEYS is the key your client sends to PrivAiTe (pick any value); OPENAI_API_KEY is your real provider key, which stays in the container and never reaches your client. This exposes gpt-4o-mini and gpt-4o; for any other provider (Ollama, Azure, anything LiteLLM supports), mount a config: configuration.

pip:

pip install privaite
# One spaCy model per scanned language; the default preset scans EN + FR.
python -m spacy download en_core_web_lg && python -m spacy download fr_core_news_md

cat > privaite.yaml <<'EOF'
providers:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: ${OPENAI_API_KEY}
pii:
  enabled: true
  preset: onnx    # or "light": faster, no model download, classic PII only
EOF

PRIVAITE_API_KEYS=change-me python -m privaite --config privaite.yaml

Connect: point any OpenAI-compatible client at http://localhost:8400/v1 with the key change-me. For Open WebUI: Admin → Settings → Connections → OpenAI API, URL http://localhost:8400/v1 (or http://host.docker.internal:8400/v1 if Open WebUI runs in Docker), key = your PRIVAITE_API_KEYS value. Client snippets (curl, Python, Node) are in examples/. Prefer no separate proxy? Use the in-process Open WebUI filter.

Prove it on your machine

examples/demo_tool_call_leak.py starts a fake provider that records what it receives, then sends the same agent request without and with PrivAiTe. Nothing leaves 127.0.0.1:

python examples/demo_tool_call_leak.py

 WITHOUT PrivAiTe, the provider receives:
  {'to': '[e-mail]', 'body': "... Card on file: 4111 1111 1111 1111."}
 WITH PrivAiTe, the same request arrives as:
  {'to': '<EMAIL_ADDRESS_1>', 'body': "... Card on file: <CREDIT_CARD_1>."}
 and the client still gets the real values back, restored in the reply.

The first panel is also what text-only guardrails forward: they scrub message text but pass tool-call JSON through untouched. To audit it on your own data, see verify what gets redacted.

Benchmark

Measured on 120 real documents from the open AI4Privacy pii-masking-200k dataset (458 PII items, labeled by 10 independent auditor agents and cross-checked against the dataset's own mask) across DE, EN, FR, IT, plus 14 clean documents for false positives.

Solution	Recall (span)	Recall (strict)	False positives	Tool-call protection
onnx (default)	84.5%	80.6%	2 / 14	100%
light (full Presidio)	62.4%	57.9%	3 / 14	100%
LiteLLM Presidio guardrail	70.3%	65.3%	3 / 14	0.0%
LLM Guard (Anonymize)	76.9%	74.9%	5 / 14	0.0%

The competitor rows are the REAL integrations at their genuine best, not a strawman baseline. Both scan message text but neither parses tool-call JSON, so 100% of the same PII survives inside a tool-call argument, even values they detect and scrub in plain text.

Read the 100% precisely, it is structural, not absolute: of the PII PrivAiTe detects in plain text, 100% is also removed from tool-call JSON. End to end, its tool-call leak equals its detection misses (15.5% on this corpus with the onnx preset), the same misses flat text has.

Two honesty notes, both favoring caution. LLM Guard's detection model is fine-tuned on the exact dataset behind this corpus, so its recall here is optimistic; PrivAiTe's default model is not (OpenAI's model card states it did not train on it). An out-of-distribution cross-check on two independent corpora confirms the default generalizes: ~84% held on Gretel finance text while the AI4Privacy-tuned model drops to ~62% (OOD_COMPARISON.md).

Per-language and per-entity tables, competitor configs, methodology, reproduction: privaite-bench. Feature comparison: docs/comparison.md.

Presets

Preset	What runs	Recall*	False positives	Latency	Secrets
onnx (default)	Presidio + Privacy Filter	84.5%	2 / 14	~0.5s	yes
light	Presidio only	62.4%	3 / 14	~60ms	no
max	onnx + GLiNER	higher OOD	more	~0.7s	yes

*Span recall on the AI4Privacy benchmark above. max adds GLiNER (trained on data independent of AI4Privacy): on out-of-distribution corpora it raises recall by several points at the cost of more false positives and a torch dependency (pip install 'privaite[gliner]'); with it selected but not installed, the proxy fails at startup with an install hint rather than silently degrading.

onnx = maximum coverage: secrets, passwords, API keys, unusual names, addresses. light = fastest, near-zero false positives, classic PII only, no addresses/URLs/secrets. How the engines work and what stays off by default: docs/detection.md.

Footgun: do not pin detectors.presidio.entities to a short allowlist on the light path. It restricts detection to only those types and roughly halves recall (to ~35%). Leave entities unset; the proxy logs a warning at startup if it detects a low-recall configuration.

What it scans

Before anything is forwarded: messages[].content (plain string or multimodal text parts), tool_calls[].function.arguments and the legacy function_call.arguments (parsed as JSON, scrubbed value by value including bare numeric leaves, keys and function names intact), /v1/completions prompt, /v1/embeddings input. On the way back, values are restored in content, tool calls and reasoning traces, streaming included.

NOT scanned (know your surface): messages[].name, top-level user/metadata, tools definitions, JSON object keys. Keep PII out of those fields. Endpoints, strict mode and passthrough caveats: docs/api.md.

Threat model

PrivAiTe performs local pseudonymization, not guaranteed anonymization. Detection runs on your machine; the real ↔ placeholder mapping lives in memory only for the duration of a request and is dropped afterwards.

What it protects against: the LLM provider storing, training on, or logging your raw PII. The provider receives placeholders (<PERSON_1>, …) for everything the detector catches, across message content, tool-call arguments, and multimodal text.

What it does NOT protect against:

• PII the detector misses. Detection is statistical and never 100% (see the benchmark). A name it doesn't recognize reaches the provider. Treat the output as best-effort, not a guarantee.
• Re-identification from context. Even with names replaced, the surrounding text can stay identifying ("the CEO of <ORG_1> who resigned in March").
• A compromised local machine. The mapping and raw text live in local memory; this is not a defense against a local attacker.
• The provider correlating requests within a session.

For GDPR/HIPAA: treat this as pseudonymization + transfer minimization, not anonymization. If you need irreversible removal, use method: "redact". Audit it on your own data: docs/verify.md.

Alternatives

Keeping PII out of LLM calls is a crowded space, and PrivAiTe is not always the right pick. Based on each project's public docs as of June 2026:

• LiteLLM has a built-in Presidio guardrail, the natural choice if you already run the LiteLLM proxy and want PII handling inline (there are a few open bugs around scrubbing requests and responses).
• Managed/cloud options exist too, such as Microsoft PII Shield and LangChain's gateway redaction.

Where PrivAiTe differs: it anonymizes PII inside tool-call arguments and multimodal content, not just message text (LangChain's gateway docs, for instance, note that tool-call arguments are not scanned), it restores the original values in the response, and it ships a reproducible benchmark. If your traffic is agentic or multimodal, that gap is the reason this exists.

Integrations

• Open WebUI filter (setup, hub listing): an Open WebUI Filter Function running the engine in-process, no separate proxy. Admin Panel → Functions → paste integrations/openwebui/privaite_filter.py, enable, pick preset and languages in its valves. Covers message text, tool calls and multimodal.
• LiteLLM guardrail (setup): a custom guardrail for teams already on the LiteLLM proxy. Mount integrations/litellm/privaite_guardrail.py next to your config.yaml to anonymize requests and restore responses inline, including tool-call arguments, which LiteLLM's built-in Presidio guardrail does not scan.

Docs

• How detection works: the two engines, what each catches, what stays off by default, known limitations
• Configuration reference: providers, Docker with custom config, anonymization methods, block_entities, custom patterns, languages, pinned model revisions
• API reference: endpoints, the exact scanned/unscanned surface, strict mode, passthrough caveats
• Verify what gets redacted: audit the proxy on your own data, dry-run inspect endpoint
• Feature comparison and the reproducible benchmark
• Changelog

Development

git clone https://github.com/crp4222/PrivAiTe && cd PrivAiTe
pip install -e ".[dev]"
python -m spacy download en_core_web_lg && python -m spacy download fr_core_news_md

cp .env.example .env                                    # keys
cp config/privaite.example.yaml config/privaite.yaml    # providers
python -m privaite --reload                             # dev mode (auto-reload)

python -m pytest tests/ -v

License

BSD 3-Clause. See LICENSE.