langchain-placeholder-guard
A model-agnostic LangChain agent middleware that deterministically detects unfilled template placeholders and forbidden internal terms in model output and outgoing tool arguments, then retries, blocks, or flags. No extra model call.
Why
Agents drafting user-facing content regularly ship [Client Name], {{first_name}}, <AMOUNT>, ${date}, "INSERT LINK HERE", or internal terms such as tool names and routing labels. Prompt instructions reduce this but do not eliminate it; LLM-judge checks cost a model call and are non-deterministic.
Install
pip install langchain-placeholder-guard
Usage
from langchain.agents import create_agent
from langchain_core.tools import tool
from langchain_placeholder_guard import PlaceholderGuardMiddleware
@tool
def send_email(to: str, body: str) -> str:
"""Send an email."""
return "sent"
agent = create_agent(
model="openai:gpt-4o", # any chat model
tools=[send_email],
middleware=[
PlaceholderGuardMiddleware(
check_tool_args=["send_email"], # also scan outgoing email args
forbidden_terms=["send_email", "[INTERNAL]"],
),
],
)
What a retry looks like
model call 1: [Human: "Write a greeting"]
-> AIMessage("Dear [Client Name], about 5 days.")
model call 2: [Human: "Write a greeting",
Human: "Your previous response contained unfilled placeholders ...
- '[Client Name]' in text: '...Dear [Client Name], about 5 days....'"]
-> AIMessage("Dear Ann, about 5 days.")
The bad message is not replayed, because a message with tool calls cannot be sent back without its tool results.
Block or flag instead of retrying
# Raise PlaceholderLeakError(findings) on the first finding.
PlaceholderGuardMiddleware(strategy="block")
# Return the response unchanged, with findings in response_metadata["placeholder_guard"].
PlaceholderGuardMiddleware(strategy="flag")
Tuning the patterns
from langchain_placeholder_guard import OPTIONAL_PATTERNS
PlaceholderGuardMiddleware(
extra_patterns=OPTIONAL_PATTERNS, # also flag TODO/TBD/FIXME and {single_brace}
ignore=["[Note]", "[Important]"], # known-safe labels
detectors=[my_domain_check], # (str) -> list[Finding]
)
Combine with HumanInTheLoopMiddleware on send_email so reviewers see drafts whose placeholders are already corrected.
How it works
- Scans the final text of responses without tool calls and, optionally, the string arguments of tool calls (recursively; locations look like
tool:send_email:recipients[0].name) before the tool runs. - Strategies:
retry(default): re-calls withoriginal messages + [nudge]. The nudge quotes each finding with surrounding context.block: raisesPlaceholderLeakError(findings).flag: returns a copy of the response with findings inresponse_metadata["placeholder_guard"].
- Fenced and inline code is ignored by default.
Default patterns: double_brace ({{ x }}), dollar_brace (${x}), bracket_label ([Client Name]), angle_label (<AMOUNT>, common HTML tags excluded), insert_here. Opt-in via extra_patterns=OPTIONAL_PATTERNS: todo_marker, single_brace.
The detector is exported:
from langchain_placeholder_guard import find_placeholders
find_placeholders("Dear [Client Name], total {{ amount }}")
# -> two Finding objects: kind='double_brace' match='{{ amount }}'
# kind='bracket_label' match='[Client Name]'
# Each has .kind, .match, .location ('text' or 'tool:<name>:<arg path>') and .snippet.
Configuration
| Constructor arg | Default | Description |
|---|---|---|
strategy |
"retry" |
"retry", "block" or "flag". |
max_retries |
2 |
Only for retry. |
check_final_text |
True |
Scan final response text. |
check_tool_args |
False |
True for all tools, or a list of tool names. |
patterns |
defaults | Named regexes replacing the defaults. |
extra_patterns |
None |
Named regexes merged in. |
forbidden_terms |
() |
Strings (case-insensitive, word-bounded) or compiled regexes. |
flag_bound_tool_names |
False |
Treat bound tool names as forbidden in final text only. |
ignore |
() |
Exact matches never reported (for example "[Note]"). |
ignore_code_blocks |
True |
Skip fenced and inline code. |
detectors |
() |
Custom (str) -> list[Finding] callables. |
nudge |
see DEFAULT_PLACEHOLDER_NUDGE |
Text containing {findings}. |
on_exhausted |
"return" |
"return" returns the last response unchanged, "error" raises PlaceholderLeakError, any other string is returned to the user as the reply. |
Ordering
Middleware order is outermost first.
- Place inside middleware that trims message history, so the nudge is not trimmed away.
- Put a
ModelCallLimitMiddlewareoutermost; retries across several guards multiply model calls.
Limitations
- Regex heuristics: tune
ignoreandpatternsfor your domain.[Note]-style labels in legitimate prose are a known false-positive class. - Streaming: the rejected attempt may already have streamed. For strict guarantees on final text, do not stream raw tokens to end users, or rely on tool-argument checks for outbound messages.
- Does not validate that a filled value is correct, only that it is not a placeholder.
- Not a PII tool. Combine with
PIIMiddlewareif needed.
Development
pip install -e ".[dev]"
pytest
License
MIT
Metadata
Release files for langchain-placeholder-guard 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| langchain_placeholder_guard-0.1.0.tar.gz | 12.7 kB | Without provenance | Details for langchain_placeholder_guard-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| langchain_placeholder_guard-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for langchain_placeholder_guard-0.1.0-py3-none-any.whl |
Total release size: 24.1 kB
Release files / langchain_placeholder_guard-0.1.0.tar.gz
| Download URL | langchain_placeholder_guard-0.1.0.tar.gz |
|---|---|
| Size | 12.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9c195bbef19f84e72e93cdd3a5e1aacd591df8c1b3723dacfbb45120a719079a
|
|
BLAKE2b-256 checksum How to use checksums |
9d62cf484155ca1495d196bc86f5bb839c0ea3c3bf0e6daae80ac3ff612392a7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.10
|
Release files / langchain_placeholder_guard-0.1.0-py3-none-any.whl
| Download URL | langchain_placeholder_guard-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8cac426aba107113ac2a800c5b3d62a3ee6e346022e6858d8243bf41bd960d6f
|
|
BLAKE2b-256 checksum How to use checksums |
2ecd33df7d1b46304006825338560b158b79a1f2ddc4d52db86c6f145dec5ae9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.10
|
Security for langchain-placeholder-guard 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.