langchain-redefiner
Personal-data redaction and entity extraction for LangChain, using ReDefiNER: an open model (0.8B parameters, Apache 2.0) that finds what a rulebook written in plain English defines. It runs locally on CPU or GPU, so text is redacted before it leaves your machine. English only.
ReDefiNERPIIMiddleware: agent middleware that redacts personal data in user messages (and optionally model output and tool results) before the model sees it. It is LangChain's ownPIIMiddlewarewith ReDefiNER as the detector, so one middleware covers every type in the rulebook.redefiner_detector: the detector on its own, forPIIMiddleware(..., detector=...).ReDefiNERRedactorandReDefiNEREntityExtractor: document transformers that mask documents, or add the entities they mention to their metadata.
By default the PII rulebook of jungroo-ai/redefiner-pii is used
(20 types: names, contact details, addresses, dates, ID and card numbers and more). Pass your own rulebook to redact
by your own policy.
Install
pip install langchain-redefiner
The first call downloads the model weights (about 0.8 GB) from Hugging Face.
Agent middleware
from langchain.agents import create_agent
from langchain_redefiner import ReDefiNERPIIMiddleware
agent = create_agent(
"openai:gpt-5",
tools=[...],
middleware=[ReDefiNERPIIMiddleware()],
)
agent.invoke({"messages": [{"role": "user", "content": "Refund Meera Nair, email meera.nair@example.org."}]})
# the model receives: "Refund [REDACTED_PERSON], email [REDACTED_EMAIL]."
Options:
| Option | Default | Meaning |
|---|---|---|
model |
"jungroo-ai/redefiner-pii" |
A Hugging Face repo or local folder |
rulebook |
the model's default | Rulebook text, a path to a rulebook file, or a dict |
strategy |
"redact" |
LangChain's strategies: "redact", "mask", "hash" or "block" |
types |
all | Only these rulebook types, e.g. ["PERSON", "EMAIL"] |
apply_to_input |
True |
Redact user messages before the model call |
apply_to_output |
False |
Redact the model's messages |
apply_to_tool_results |
False |
Redact tool results |
min_confidence |
0.0 |
Ignore spans below this confidence |
none_bias, threshold, max_span_tokens |
model defaults | ReDefiNER settings; see its README |
device |
GPU if available | "cpu", "cuda" or "cuda:N" |
Each match's type is the rulebook type in lower case ([REDACTED_PERSON]). Emails, IP addresses, URLs and card numbers
use LangChain's own type names, so strategy="mask" handles them as it does its built-in types
(meera.nair@****.org).
To use LangChain's middleware directly:
from langchain.agents.middleware import PIIMiddleware
from langchain_redefiner import redefiner_detector
PIIMiddleware("redefiner", detector=redefiner_detector(types=["PERSON"]), strategy="redact")
Your own policy
Write the types you want masked, with a one-line description each, and anything that must stay:
policy = """[TYPE] CUSTOMER_NAME
[DESCRIPTION] The name of a customer. Our own staff, who sign as "Agent <name>", are not customers.
[TYPE] ACCOUNT_NUMBER
[DESCRIPTION] A customer account number, such as "AC-449120".
"""
middleware = ReDefiNERPIIMiddleware(rulebook=policy)
# "Customer Daniel Brandt (account AC-771204) asked for a refund. Agent Rosa Diaz approved it." reaches the model as
# "Customer [REDACTED_CUSTOMER_NAME] (account [REDACTED_ACCOUNT_NUMBER]) asked for a refund. Agent Rosa Diaz approved it."
See the rulebook guide. Try a policy on a few dozen of your own conversations before relying on it.
Documents
from langchain_core.documents import Document
from langchain_redefiner import ReDefiNEREntityExtractor, ReDefiNERRedactor
docs = [Document(page_content="Hi, this is Meera Nair from Brightwater Dental. Call me on 020 7946 0155.")]
ReDefiNERRedactor().transform_documents(docs)[0].page_content
# 'Hi, this is [PERSON] from [ORGANIZATION]. Call me on [PHONE].'
rulebook = """[TYPE] PERSON
[DESCRIPTION] A person's name.
[TYPE] ORGANIZATION
[DESCRIPTION] A company or institution.
"""
ReDefiNEREntityExtractor(rulebook=rulebook).transform_documents(docs)[0].metadata
# {'entities': {'PERSON': ['Meera Nair'], 'ORGANIZATION': ['Brightwater Dental']}}
- The redactor records what it masked in
metadata["redefiner_redactions"]: type, confidence and offsets, but not the masked text (unlesskeep_masked_text=True). ReDefiNERRedactor().redact_textmasks a single string; wrap it inRunnableLambdato use it in a chain.ReDefiNEREntityExtractor(..., flat=True)writes one key per type ("entity_PERSON": "Meera Nair") for vector stores that only accept flat metadata.
Limits
- Not the only safeguard. The model misses some personal data and marks some text that is not personal. Check it on your own data.
- Speed. On CPU the first call reads the rulebook (several seconds for the PII rulebook) and is then cached; later short messages take under a second. Each redacted message is one model call.
- English only.
Development
pip install -e ".[dev]"
pytest # fast tests with a stand-in model
pytest -m slow # the real model on CPU
Licence
Apache License 2.0. See LICENSE. Made by Jungroo AI Labs.
Metadata
Release files for langchain-redefiner 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| langchain_redefiner-0.1.0.tar.gz | 16.1 kB | Without provenance | Details for langchain_redefiner-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| langchain_redefiner-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for langchain_redefiner-0.1.0-py3-none-any.whl |
Total release size: 29.4 kB
Release files / langchain_redefiner-0.1.0.tar.gz
| Download URL | langchain_redefiner-0.1.0.tar.gz |
|---|---|
| Size | 16.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
72b2b239e762f5030bb933f5b97d0e7172fddeb125619576a913088a9b6a5447
|
|
BLAKE2b-256 checksum How to use checksums |
7a40d407c949c6ffa6e92b3ad64f79e14832e3e9906310dd5e76c36a4dbf3728
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Release files / langchain_redefiner-0.1.0-py3-none-any.whl
| Download URL | langchain_redefiner-0.1.0-py3-none-any.whl |
|---|---|
| Size | 13.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d4573c7548da173c4b371fe8d188eb38cbbe2df0e94f1b3e6022a8e388b325d8
|
|
BLAKE2b-256 checksum How to use checksums |
070ba2c45589ea70f31a0ec69b183f0d70a86994401400ff8570610ac7dcca0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.6
|
Security for langchain-redefiner 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.