langchain-sct

Reversible PII pseudonymization for LangChain 1.0 — a keyed, DSGVO/BaFin-grade superset of the built-in PIIMiddleware.

LangChain's own privacy tools are destructive or fake-reversible: PIIMiddleware's redact/mask/hash strategies are one-way, and PresidioReversibleAnonymizer keeps a plaintext mapping.json. So the agent's final answer contains [REDACTED_email] instead of the real name/IBAN.

langchain-sct closes the loop. It pseudonymizes PII into reversible SCT tokens before the model call and re-identifies them after — the LLM never sees plaintext PII, yet the user gets a fully usable, re-identified answer. Keying is AES-256-GCM / FF3-1 FPE with tenant key custody and audit, server-side.

Install

pip install langchain-sct

Flagship: SCTPseudonymizationMiddleware

One line into create_agent gives any agent reversible PII protection with zero user code change:

from langchain.agents import create_agent
from langchain_sct import SCTPseudonymizationMiddleware

agent = create_agent(
    "openai:gpt-5.5",
    tools=[...],
    middleware=[SCTPseudonymizationMiddleware(api_key="sct_...")],
)

# The LLM sees "<PERSON_0> at <EMAIL_1>"; the caller gets the real values back.
agent.invoke({"messages": [("user", "Email Max Mustermann at max@example.com")]})

	built-in PIIMiddleware	SCTPseudonymizationMiddleware
Strategy	redact / mask / hash	pseudonymize (reversible)
Final answer	[REDACTED_email]	real value re-identified
Reverse the loop	impossible (one-way)	keyed de-pseudonymize
Entity coverage	5 regex types (email, credit_card, ip, mac_address, url)	13 DE/EU NER + regex types
Key custody	—	AES-256-GCM / FF3-1, audited

Configuration

SCTPseudonymizationMiddleware(
    api_key="sct_...",                     # or set SCT_API_KEY
    strategy="pseudonymize",               # default reversible
    strategies={"CREDIT_CARD": "block"},   # per-type override (fail-fast)
    encryption_method="fpe-ff1",           # format-preserving tokens
    apply_to_input=True,
    apply_to_tool_results=False,
)

strategy="block" (or a per-type block) raises SCTPIIDetectionError in before_model — the run fails before any content reaches the LLM.

Portable LCEL: with_sct_pseudonymization

Middleware only fires inside create_agent. For plain LCEL chains (any provider, any LangChain 0.x/1.x), wrap the chain instead:

from langchain_sct import with_sct_pseudonymization

chain = with_sct_pseudonymization(prompt | llm | StrOutputParser(), client=sct)
answer = chain.invoke("Please email Max Mustermann at max@example.com")

The reversible token map is carried through the chain as the encryption_key via a RunnableParallel fork — no plaintext mapping table ever leaves the SCT boundary. sct_pseudonymize / sct_reidentify are exported for hand-composed chains.

Token compression: SCTCompressionMiddleware

The second differentiator — deterministic tool-output compression. LangChain's own options for bulky observations are blind truncation (dumb) or a summarizer-LLM call (an extra, non-deterministic model round-trip, explicitly risky for finance/legal/health). SCT compresses via POST /tokenizer/compress: format-aware, deterministic, never_worse (tiktoken-verified never to cost more tokens than the raw input), zero extra LLM call.

from langchain_sct import SCTCompressionMiddleware

agent = create_agent(
    "openai:gpt-5.5",
    tools=[search, run_tests],
    middleware=[SCTCompressionMiddleware(api_key="sct_...", min_chars=200)],
)

It overrides wrap_tool_call: the tool runs, then its ToolMessage content is compressed on the way back into context. Each compressed message carries its per-request savings_pct / tier in response_metadata["sct_compression"] — the billing envelope travels with the message, not on the shared instance.

RAG: SCTRetriever + SCTPseudonymizingDocumentTransformer

Extend "PII never enters context" to retrieval — RAG's biggest leak surface. Retrieved chunks are pseudonymized before they reach the prompt; pair with the middleware/anonymizer to re-identify the answer with the stamped key.

from langchain_sct import SCTRetriever

retriever = SCTRetriever.from_api_key(vectorstore.as_retriever(), api_key="sct_...")
# Each returned Document has token-only page_content + metadata["sct_encryption_key"].

SCTPseudonymizingDocumentTransformer is usable standalone at ingest time (transform_documents / atransform_documents).

Presidio migration: SCTReversibleAnonymizer

Same anonymize() / deanonymize() surface as langchain_experimental's PresidioReversibleAnonymizer, but reversibility is keyed crypto — no plaintext mapping.json crown-jewel to leak, and FF3-1 tokens stay format-preserving. Migrate by changing one import:

from langchain_sct import SCTReversibleAnonymizer

anonymizer = SCTReversibleAnonymizer(api_key="sct_...")
clean = anonymizer.anonymize("Email Max Mustermann at max@example.com")
restored = anonymizer.deanonymize(llm_answer)

Billed endpoint: ChatSCT + SCTUsageCallback

ChatSCT is a BaseChatOpenAI subclass pinned to the SCT api-gateway's OpenAI-compatible /v1 surface — tenant/API-key headers injected, metering on by default. Requires the openai extra:

pip install "langchain-sct[openai]"

from langchain_sct import ChatSCT

llm = ChatSCT(model="gpt-5.5", api_key="sct_...", tenant_id="acme")

SCTUsageCallback.on_llm_end feeds each call's usage_metadata into an SCT metering sink (wire your Stripe push there). A custom gateway base_url disables OpenAI's streamed usage, so the callback carries a tiktoken fallback for exact local counts.

Async

Every surface has a native async path: the middleware implements abefore_model / aafter_model / awrap_tool_call on AsyncSCTClient, the runnables accept an async_client= for ainvoke / abatch, and the anonymizer and document transformer expose aanonymize / adeanonymize / atransform_documents.

How it works

• before_model → POST /pseudonymize (auto_detect_pii=true); the returned encryption_key is carried in agent state (a private field), not on the middleware instance, so concurrent runs never share keys.
• after_model → POST /de-pseudonymize with that key, restoring the real values in the returned AIMessage.
• One key is reused across a conversation (bring-your-own-key on later turns) so re-identification is a single call.

Benchmarks

The point isn't throughput — it's whether the agent's final answer is usable. All three tools keep plaintext PII out of the model context; only SCT gives the caller a re-identified answer back, and only SCT does it without a plaintext mapping table.

Input in every row: "Email Max Mustermann at max@example.com about IBAN DE89370400440532013000".

	What the LLM sees	What the caller gets back	Reversible?	Secret at rest
built-in PIIMiddleware (redact)	Email [REDACTED_email]...	Email [REDACTED_email] about [REDACTED_...] — destroyed	no (one-way)	none needed
built-in PIIMiddleware (hash/mask)	Email a1b2c3… / ****@****	still hashed/masked — not usable	no	none
PresidioReversibleAnonymizer	Email <PERSON> at <EMAIL_ADDRESS>	real values if you call deanonymize()	yes	plaintext mapping.json on disk — the crown jewel
SCTPseudonymizationMiddleware	Email <PERSON_0> at <EMAIL_1> about <IBAN_2>	Email Max Mustermann at max@example.com about DE89370400440532013000 — fully re-identified, automatically	yes (keyed)	ciphertext only; key custody + audit server-side, no plaintext map

Concretely, side by side:

# built-in PIIMiddleware — the answer is dead on arrival
create_agent(model, middleware=[PIIMiddleware("email", strategy="redact")])
# -> AIMessage("I've emailed [REDACTED_email].")   # useless to the user

# PresidioReversibleAnonymizer — reversible, but the mapping is plaintext on disk
anon = PresidioReversibleAnonymizer()
anon.anonymize("max@example.com")     # writes {"<EMAIL_ADDRESS>": "max@example.com"} to mapping.json
# a leaked mapping.json re-identifies every past run at once

# SCTPseudonymizationMiddleware — reversible AND keyed, no plaintext map, one line
create_agent(model, middleware=[SCTPseudonymizationMiddleware(api_key="sct_...")])
# -> AIMessage("I've emailed Max Mustermann at max@example.com.")  # usable, auto re-identified

Token-compression benchmark

SCTCompressionMiddleware is measured against LangChain's two options for bulky tool observations:

Approach	Determinism	Extra LLM call	Guarantee
Blind truncation	deterministic	none	loses information silently
SummarizationMiddleware (summarizer LLM)	non-deterministic	yes (extra round-trip)	none — can hallucinate, risky for finance/legal/health
SCTCompressionMiddleware	deterministic	none	never_worse — tiktoken-verified never to cost more tokens than the raw input

Each compressed ToolMessage reports its realized savings_pct / tier in response_metadata["sct_compression"], so the win is measured per call, not assumed.

DSGVO / GDPR nuance (read before you ship to a regulated tenant)

Reversibility is the feature — and the caveat. Be precise about what SCT does and does not change legally:

• Pseudonymized ≠ anonymized. Under Art. 4(5) DSGVO / GDPR, reversibly pseudonymized data is still personal data (Recital 26) because the key can restore it. SCT reduces risk and enforces data minimization toward the LLM provider; it does not take the data out of scope. Do not market it as "anonymization." If you need true anonymization (out of scope), use a destructive strategy (PIIMiddleware redact/hash) instead — accepting that the answer is no longer re-identifiable.

• Key custody is the control that matters. The whole security argument over Presidio's mapping.json is that SCT never persists a plaintext token→value map: re-identification requires the tenant key, held server-side with audit. Treat the SCT API key (sct_...) and the per-conversation encryption_key as the crown jewels — scope them per tenant, rotate them, and keep them out of logs and out of the model context (the middleware already keeps the key in private agent state, never on the instance and never in a message).

• Audit + purpose limitation. Every de-pseudonymize is a re-identification event; log who triggered it and why. Set retention on encryption keys — once a conversation's key is destroyed, its tokens are cryptographically irreversible, which is a clean deletion story for erasure requests (Art. 17).

• Deny/block on low-confidence entities for regulated tenants. Auto-detect can miss or mis-type an entity. For BaFin/finance/health tenants, prefer a block strategy on the high-stakes types so an undetected or low-confidence hit fails the run rather than leaking to the model:

  SCTPseudonymizationMiddleware(
      api_key="sct_...",
      strategies={"IBAN": "block", "CREDIT_CARD": "block", "HEALTH": "block"},
  )
  

  block raises SCTPIIDetectionError in before_model — nothing reaches the LLM. Fail closed, not open.

• Not legal advice. This documents SCT's technical behavior; your DPO owns the DSGVO assessment (DPIA, records of processing, DPA with the model provider).

License

MIT © SIMO GmbH — see LICENSE.