Sether (Python)
Keep the context. Protect the details.
Your AI can help write the email. It doesn't need your actual email address.
Sether is a small, deterministic engine for the boundary between your app and any model. It finds personal data and secrets in text and swaps them for tokens or realistic decoys before the text leaves your process. When the reply comes back, it puts the originals back. It's streaming-safe and JSON-safe, the vault stays on your side, and the library makes no network calls of its own.
pip install sether
Live sandbox · Python docs · TypeScript package · Changelog · MIT
Named for the Hebrew sether (סֵתֶר), "the hiding place" (Psalm 32:7). The name is also the spec.
Why this exists, and why now
AI got useful by reading the real thing. Support tickets, logs, CRM rows, contracts. People paste the whole thing because they want a good answer. The task needs help. The details hitch a ride.
- 77% of employees who use GenAI paste data into it, and 22% of those pastes contain personal or payment-card data (LayerX report, via The Register, Oct 2025).
- Harmonic Security counted 579,113 sensitive-data exposures across 22.5 million enterprise AI prompts and uploads in 2025 (Harmonic, Jan 2026).
- 92% of organisations that had an AI-related security incident lacked proper AI access controls (IBM Cost of a Data Breach 2026).
Agents raised the stakes. A chatbot reads your data. An agent acts on it: it
updates the CRM, sends the email, files the ticket. The tool needs the real
address back, so one-way masking breaks the workflow, and a workflow that breaks
tends to get switched off. Hiding the value is the easy half. Giving it back
safely is the hard half. We learned that one the honest way: 0.4.0 (0.8.0 in
TypeScript) fixed our own restore path, where a value restored into tool-call
JSON as plain text could add a bcc recipient.
The rules already apply.
- An LLM API provider typically acts as your processor, and GDPR Article 28 expects processors that give sufficient guarantees, bound by a contract.
- OWASP lists LLM02:2025 Sensitive Information Disclosure in its Top 10 for LLM applications.
- In Nigeria, the NDPC fined MultiChoice ₦766.2 million in July 2025, citing privacy breaches and unlawful cross-border transfers of personal data (ICIR).
Detection is becoming free, and that's good news.
- LangChain's PIIMiddleware can redact, mask, hash or block.
- n8n's Guardrails node swaps PII for placeholders.
- OpenAI released an open-weight Privacy Filter (Apache-2.0) in April 2026.
- Presidio, originally Microsoft's and now community-maintained, is a mature detector.
Sether focuses on the part around detection: deterministic, streaming-safe replacement, a vault you own, and a restore that can't rewrite your JSON. If you prefer another detector, plug it in as a custom detector.
How the engine works
A stand-in goes out. The original comes back. Four steps, all in your process:
| Step | What happens | The detail that matters |
|---|---|---|
| 1. Detect | Detector packs scan the text: emails, phones, cards, SSNs, IPs, IBANs by default; secrets and labelled identity fields when you opt in. | Every detector is deterministic (pattern plus validator: Luhn, mod-97, libphonenumber, SSA rules). The same input gives the same output. The patterns are the TypeScript engine's, and every one is ReDoS-checked in that repo's CI. |
| 2. Replace | Each match becomes a typed token such as <EMAIL_9f1c…>, or a realistic decoy from an AliasVault. |
The original goes into your vault (in-memory LRU by default: 10,000 entries, 1-hour TTL). It never goes into the token or an audit event. |
| 3. Stream | Redaction runs over sync or async iterables of text, so token-by-token LLM traffic works. | The stream holds back a 256-byte tail (and long secrets up to 8 KB), so a value split across two chunks is still caught. Property-based tests split text at random boundaries and check the round trip. |
| 4. Restore | Tokens or decoys in the reply are swapped back for the originals. | For JSON (tool calls, JSON mode), restore_json restores value by value and never touches keys. strict=True refuses tokens the vault never issued. |
your app ──► detect ──► replace ──► model sees <EMAIL_9f1c…>
▲ │
│ your vault (originals)
│ │
└──── restore ◄─────────┴──── reply mentions <EMAIL_9f1c…>
What Sether is, and isn't
- It is a boundary, not a model. No ML runs on the hot path, so results are reproducible and testable. For free-text names, pair it with your own NER as a custom detector.
- It keeps your context. The model still gets the task, the structure and a
stand-in it can reason about. "Draft a welcome email to
<EMAIL_1>" is still a perfectly good prompt. - It is inspectable. MIT, a wheel of under 60 KB, one runtime dependency
(
phonenumbers), published from CI with PyPI Trusted Publishing. Read it, fork it, audit it. - It is honest about misses. Detection is pattern-based. It will miss things it has no pattern for, and dense numbers can trip the card check. Review both kinds of error on your own data. Honest limitations lists the known ones.
- It isn't a compliance certificate. It's a technical control you can point to, and a record of what it replaced. What you claim to a regulator is still your call.
The Sether family
The same engine runs in four places:
| Where | What |
|---|---|
| Your Python app | This package, sether: sync and async streaming, plus httpx, ASGI, WSGI, OpenAI and Anthropic integrations |
| Your TypeScript app | @raeven-co/sether on npm, the original engine this package ports 1:1 |
| Your browser | Sether Shield for Chrome: detection, masks and restore in AI chat boxes, all local. Decoys, watchlists and custom rules are in Pro. |
| Your editor | Sether for VS Code: scan, Copy Scrubbed, Paste with Originals Restored |
| Your agent's tool calls | Gateway Guard (beta, self-hosted): an OpenAI-compatible hop that refuses tool calls whose tokens weren't issued in that run |
The animation at the top shows Sether Shield. It's an illustration with fictional data.
Install
pip install sether
Requires Python 3.9+. The phone detector uses phonenumbers
(installed automatically). Integration extras are optional:
pip install "sether[openai]" # wrap_openai
pip install "sether[anthropic]" # wrap_anthropic
pip install "sether[httpx]" # wrap_httpx
pip install "sether[all]" # all of the above
The ASGI and WSGI middlewares have no extra dependency.
60-second quickstart
from sether import Sether
sether = Sether()
# Outgoing: redact before sending to the LLM.
safe = sether.redact_sync("my email is alice@example.com")
# -> "my email is <EMAIL_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx>"
# Incoming: restore before showing the user.
back = sether.restore_sync(safe)
# -> "my email is alice@example.com"
The same Sether instance shares its vault between redaction and restoration,
which is how the round-trip identity is preserved.
Security: never call
restore_sync()on serialized JSON (tool-call arguments, JSON-mode replies, JSON response bodies). Restore is plain text substitution: an original value containing"or\can add keys to the document or break it. Userestore_json/restore_json_value, see Never restore serialized JSON as text.
Streaming (sync and async)
sether = Sether()
# Synchronous: any iterable of text chunks.
def chunks():
yield "Contact alice@"
yield "example.com or call "
yield "+1 415 555 2671."
redacted = "".join(sether.redact_stream(chunks()))
restored = "".join(sether.restore_stream([redacted]))
# Asynchronous: any async iterable (e.g. an LLM token stream).
async def run(llm_stream):
async for piece in sether.aredact_stream(llm_stream):
await forward_to_model(piece)
The redact stream holds back safe_distance_bytes (default 256) at the tail of
each chunk so a PII pattern crossing a chunk boundary is still detected when the
next chunk arrives. A long whitespace-free value (a JWT, an API key) is held
back up to max(safe_distance_bytes * 4, 8192) bytes so it is never emitted
partially across a boundary. For values larger than that bound, raise
safe_distance_bytes or use redact_sync on complete payloads. This round-trip
identity is verified by property-based tests over 60 random chunk partitions.
Never restore serialized JSON as text
restore_sync, the restore streams and sether.restore_sync() are plain
text substitution: each original value is inserted exactly as it was,
unescaped. If the text is serialized JSON, such as LLM tool-call arguments, a
JSON-mode reply or a JSON response body, an original containing " or \
stops being data inside a string and becomes part of the document:
# Tool-call arguments from the model reference a token...
args = '{"to": "<EMAIL_...>", "subject": "Hello <NAME_...>"}'
# ...whose original is: Alice", "bcc": "attacker@example.com
json.loads(restore_sync(args, vault))
# -> {'to': 'bob@example.com', 'subject': 'Hello Alice', 'bcc': 'attacker@example.com'}
Built-in detectors can capture a trailing backslash (password: hunter2\),
which breaks the document the same way; custom detectors and vaults filled from
other systems can carry any character. Restore JSON value by value instead:
from sether import restore_json, restore_json_value
# Serialized JSON in, serialized JSON out: each string value is restored and re-escaped.
safe_args = restore_json(tool_call.function.arguments, vault, strict=True)
# or, on the facade: sether.restore_json(tool_call.function.arguments, strict=True)
# Already parsed, e.g. an Anthropic tool_use block's `input`.
tool_input = restore_json_value(block.input, vault, strict=True)
- Object keys are never restored. Invalid JSON raises
json.JSONDecodeError; there is no fallback to text replacement. - One level at a time: a string value that itself holds serialized JSON is restored as plain text, so parse and restore it separately.
restore_jsonre-serializes compactly; input formatting is not kept.strict=True(also accepted byrestore_sync) raisesUnresolvedTokenErrorwhen anything token-shaped would be left in place: an invented or expired token, a case-altered copy such as<email_...>, or a token in a key. Itstokensattribute lists them; restored originals are never included. Withoutstrict, those pass through silently.- Strict mode is not authorization. A model can still put a real token in the
wrong field (the customer's address in
bcc), so validate tool arguments before acting on them. TOKEN_REis exported if you need to scan for tokens yourself.
Sether's own restore paths restore JSON value by value. They change only the string values that hold a token and re-encode just those, so every other byte is kept (formatting, key order, escapes and numbers). Object keys are never restored.
sse_restore_iter/create_sse_restore_stream: adata:payload that parses as JSON (OpenAI and Anthropic chunks) is restored value by value, so an original containing",\or a line break can't change the JSON or break the SSE framing. Other payloads ([DONE], plain text) are restored as text, and a line break in an original continues it on a newdata:line (the client receives it as\n). Eachdata:line is handled on its own, so a JSON payload split across severaldata:lines is restored as text.wrap_httpx,SetherASGIMiddlewareandSetherWSGIMiddleware: anevent-streambody is restored perdata:payload as above. A body that parses as JSON is restored value by value, a JSON-typed body that doesn't (NDJSON, JSON Lines) line by line, and any other text body as text.wrap_openai/wrap_anthropic: when the request asks for JSON output (response_formatjson_object/json_schema, or Anthropicoutput_config.format/output_formatwithjson_schema), the reply's content is restored value by value, and left tokenized if it isn't valid JSON. Streamed JSON-mode deltas are fragments of a document, so they stay tokenized: join them and callrestore_json. Tool-call arguments stay tokenized by default. Passrestore_tool_arguments=Trueto restore non-streamingtool_calls[].function.argumentsandtool_use.inputvalue by value, and validate them before acting on them.
What is still restored as text:
- JSON-mode output streamed over SSE.
sse_restore_iter(and the middlewares on an event-stream body) keeps each chunk valid, but thedelta.contentfragments join into a second JSON document the stream can't see, so an original lands in that document unescaped. Don't SSE-restore JSON-mode streams: collect the tokenized fragments and callrestore_jsonon the assembled text. restore_sync, the restore streams andsether.restore_sync(), by design.AliasVault.restore()swaps decoys back by plain substitution too. On JSON, apply it to each parsed string value, never to the serialized document.
Detectors
By default Sether runs the basic pack. Pass an explicit list to narrow scope, or add the opt-in packs.
from sether import Sether, basic_detectors, secrets_detectors, identity_detectors
sether = Sether(detectors=[*basic_detectors, *secrets_detectors, *identity_detectors])
Basic pack (basic_detectors)
| Detector | Type | Method |
|---|---|---|
email_detector |
EMAIL |
RFC-5321-style regex. ASCII-only. |
phone_detector |
PHONE |
phonenumbers (international format). For national-format numbers use create_phone_detector(default_country="US"). |
credit_card_detector |
CC |
Bounded regex + Luhn check. |
ssn_detector |
SSN |
Regex + SSA invalid-prefix rules. |
ipv4_detector |
IPV4 |
Strict octet-bounded regex. |
ipv6_detector |
IPV6 |
Candidate regex + in-tree validator. |
iban_detector |
IBAN |
Regex + mod-97 checksum. |
Secrets pack (secrets_detectors)
aws_access_key_detector, openai_key_detector, anthropic_key_detector,
github_pat_detector (classic + fine-grained), slack_token_detector,
stripe_key_detector, jwt_detector, high_entropy_detector
(Shannon entropy >= 3.5 bits/char).
Identity pack (identity_detectors, opt-in)
Label-anchored detection for names, dates of birth, passport numbers, and
addresses. A value is redacted only when it appears with the label that
introduces it (Name:, DOB:, Passport No:, Address:) or, for a few
distinctive standalone shapes (a street line with a house number, a UK
postcode), a structure strong enough to keep false positives low. Labels are
recognised across many languages (Latin-script plus CJK, Cyrillic, Arabic), and
value capture is Unicode-aware.
Free-text NER (unlabelled names, organisations, locations in running prose) is
not covered here. The early NER package, @raeven-co/sether-ner,
is TypeScript-only for now.
Custom detectors
Anything with a type string and a detect(text) method works:
import re
from sether import DetectorMatch
class OrderIdDetector:
type = "ORDER_ID"
_re = re.compile(r"\bORD-\d{8}\b")
def detect(self, text):
return [DetectorMatch(m.start(), m.end(), m.group(0)) for m in self._re.finditer(text)]
Token vault
Tokens map back to originals through a vault. Sether ships an in-memory LRU
vault (10,000 entries, 1-hour TTL by default). Implement the Vault protocol to
change eviction, encrypt at rest, or namespace tokens per tenant.
from sether import Vault # a runtime-checkable Protocol: set/get/has/delete/size/clear
class NamespacedVault:
def __init__(self, prefix):
self._store = {}
self._prefix = prefix
def set(self, token, value): self._store[self._prefix + token] = value
def get(self, token): return self._store.get(self._prefix + token)
def has(self, token): return (self._prefix + token) in self._store
def delete(self, token): return self._store.pop(self._prefix + token, None) is not None
def size(self): return len(self._store)
def clear(self): self._store.clear()
sether = Sether(vault=NamespacedVault("tenant-42:"))
The Vault interface is synchronous: restore substitutes tokens as bytes
flow through and cannot await a lookup per token. Front an async store (Redis,
Postgres) with a synchronous in-process cache you hydrate before the restore
pass, or keep the vault in-process.
SSE / JSON-stream mode
OpenAI and Anthropic streaming responses come back as Server-Sent Events. The
SSE-aware stream redacts payloads inside data: lines while preserving the
data: / event: / id: / retry: framing and blank-line separators
verbatim.
from sether import create_sse_redact_stream, basic_detectors, MemoryVault
vault = MemoryVault()
stream = create_sse_redact_stream(basic_detectors, vault)
out = stream.feed(sse_chunk) + stream.finish()
# or the iterator helpers: sse_redact_iter(chunks, detectors, vault)
Drop-in integrations
from sether import Sether
sether = Sether()
httpx
import httpx
from sether import wrap_httpx
client = wrap_httpx(httpx.Client(), detectors=sether.detectors, vault=sether.vault)
r = client.post("https://api.example.com/v1/chat",
json={"q": "email alice@example.com"})
# The request carried <EMAIL_...>; r.text has any tokens the server echoed restored.
Works on httpx.Client and httpx.AsyncClient. Binary bodies pass through
untouched.
ASGI (FastAPI / Starlette)
from fastapi import FastAPI
from sether import SetherASGIMiddleware
app = FastAPI()
app.add_middleware(SetherASGIMiddleware, detectors=sether.detectors, vault=sether.vault)
WSGI (Flask)
from flask import Flask
from sether import SetherWSGIMiddleware
app = Flask(__name__)
app.wsgi_app = SetherWSGIMiddleware(app.wsgi_app, detectors=sether.detectors, vault=sether.vault)
OpenAI SDK
from openai import OpenAI
from sether import wrap_openai
client = wrap_openai(OpenAI(), detectors=sether.detectors, vault=sether.vault)
# Redacts messages out, restores choices back. Sync, async, and streaming clients.
Anthropic SDK
import anthropic
from sether import wrap_anthropic
client = wrap_anthropic(anthropic.Anthropic(), detectors=sether.detectors, vault=sether.vault)
# Redacts messages/system out, restores content blocks back.
The SDK wrappers are structurally typed. Sether never imports openai or
anthropic; any object matching the chat.completions.create /
messages.create shape works.
Audit events
Each redaction can be described by a structured AuditEvent that maps to the
regulation it satisfies (GDPR Art. 28, SOC 2 CC6.7, HIPAA, PCI DSS, and more,
see DEFAULT_REGULATION_MAPPINGS). The original value is never carried in an
event, only its length. The JSON wire shape matches the TypeScript package
(camelCase keys) so events are interchangeable across both.
from sether import AuditEvent, ConsoleAuditSink, MemoryAuditSink
sink = ConsoleAuditSink() # JSONL to stderr; MemoryAuditSink accumulates for tests
sink.write(AuditEvent(timestamp="...", detector="EMAIL", value_length=17, token="<EMAIL_x>"))
Honest limitations
These match the TypeScript package's documented limits:
- Email detection is ASCII-only. IDN/Unicode local parts do not match.
- IPv6
::1(loopback) is not detected. The candidate regex requires 4+ chars. Loopback is not customer PII. - Credit-card regex is permissive, then validated by Luhn. False positives in dense numeric content are possible.
- Names / DOB / passport / address are label-anchored, not free-text NER.
- Very large whitespace-free values split across chunk boundaries are held
back only up to
max(safe_distance_bytes * 4, 8192)bytes. Raisesafe_distance_bytesor useredact_syncon complete payloads. - Restore is text substitution.
restore_syncand the restore streams insert originals unescaped. The SSE, httpx, ASGI, WSGI and SDK-wrapper paths restore JSON value by value, except JSON-mode output streamed over SSE. Userestore_jsonfor your own JSON; see Never restore serialized JSON as text.
Parity with the TypeScript package
This port reproduces the TypeScript engine 1:1: the same detector
regexes (compiled with re.ASCII so \b / \d stay ASCII-only as in JS), the
same Luhn / mod-97 / SSA validation, the same overlap resolution (longest match
wins), the same <TYPE_uuid> token format, and the same safe-distance and
long-value streaming guards. 183 tests cover detectors, vault, streaming
(including a property-based chunk-partition round-trip), SSE, audit, and all
five integrations.
License
MIT (c) Godfrey Lebo / Raeven Company LTD
Reporting security issues
Email emorylebo@gmail.com. Do not file public issues for security findings.
Metadata
Release files for sether 0.4.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sether-0.4.1.tar.gz | 76.1 kB | With provenance | Details for sether-0.4.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sether-0.4.1-py3-none-any.whl | Python 3 | none | any | Details for sether-0.4.1-py3-none-any.whl |
Total release size: 136.0 kB
Release files / sether-0.4.1.tar.gz
| Download URL | sether-0.4.1.tar.gz |
|---|---|
| Size | 76.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d5bef712815b6f141cfe7df3e7462fdc931597fa2bd610d50a2952487d3ff9dd
|
|
BLAKE2b-256 checksum How to use checksums |
0a5232d68fa3bddb7ec3481d9263db1afec04b3b93ed884e45fa72123f15ce89
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logRelease files / sether-0.4.1-py3-none-any.whl
| Download URL | sether-0.4.1-py3-none-any.whl |
|---|---|
| Size | 59.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
90c2ce26f473fd1a14398e899938087da675cd29605f3874d8e2b7e82a1b1530
|
|
BLAKE2b-256 checksum How to use checksums |
0b77b1bbd1dfd9de78eae738c0e45c7d314e49b9e33213e0a32072655e280642
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logSecurity for sether 0.4.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for sether-0.4.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.