llm-input-hardening

The input boundary: what text LLMs and agents are allowed to read. Part of the Forge *-hardening family, alongside agent-api-hardening (capability boundary) and safegit (recovery boundary).

Deterministic Unicode-aware input hardening for LLM applications.

llm-input-hardening sits between untrusted text and your prompt or template layer. It bounds Unicode normalization, removes invisible or control characters that destabilize prompts, emits structured reports for logging and enforcement, and detects mixed- and whole-script confusables.

It is not prompt injection prevention. It addresses text-integrity problems, not model intent.

This README is the main entry point and quickstart. Deeper references — the full report schema, policy details, threat model, integrations, and evaluation methodology — live under docs/.

The hardening contract defines the guarantees, deliberate policy limits, and regression strategy across stages.

Install

pip install llm-input-hardening

Optional extras:

• pip install "llm-input-hardening[security]" for the confusable_homoglyphs backend
• pip install "llm-input-hardening[ftfy]" for pre-sanitize text repair

Determinism note: output is reproducible for pinned code and dependency versions. use_ftfy=True inspects the original input before running ftfy.fix_text(normalization=None) and sanitizing the repaired text. Rust owns bounded normalization, including when repair is enabled. Reports retain pre-repair security evidence; ftfy version changes can still change output and reports.

Import from llm_input_hardening.

Quickstart

from llm_input_hardening import sanitize

clean_text, report = sanitize(user_text, policy="balanced_chat")

Compact demo/report output:

from llm_input_hardening import compact_report, sanitize

clean_text, report = sanitize(user_text, policy="balanced_chat")
summary = compact_report(report, sanitized_text=clean_text)

Recursive JSON sanitization:

from llm_input_hardening import sanitize_json

clean_payload, report = sanitize_json(payload, policy="balanced_chat")

Enforcement-oriented usage:

from llm_input_hardening import sanitize_and_decide

clean_text, report, decision = sanitize_and_decide(
    user_text,
    sanitize_policy="strict_exec",
    confusables_backend="confusable_homoglyphs",
)

Execution fields need their own schema after normalization. For example, require an unchanged identifier from a known set:

clean_text, report, decision = sanitize_and_decide(
    tool_name,
    sanitize_policy="strict_exec",
    reject_on_change=True,
    validator=lambda value: value in {"search", "summarize"},
)
if decision["action"] != "allow":
    raise ValueError("tool name rejected")

validator always sees the returned text; validation errors propagate. Paths need application-specific root/allowlist checks, and code still needs parsing and sandboxing. strict_exec by itself does not authorize execution.

If your application joins untrusted fragments, inspect the final assembly:

from llm_input_hardening import sanitize_assembled

clean_text, report, decision = sanitize_assembled(
    retrieved_fragments, separator="\n\n", sanitize_policy="balanced_chat",
)
if decision["action"] != "allow":
    raise ValueError("assembled content requires review")

The helper joins raw fragments before inspection, with budgets for total characters and part count. Individually allowed fields do not authorize a later concatenation. Keep trusted message roles and instructions outside this helper.

The text API accepts at most 1,000,000 input characters by default (configurable with max_input_chars). Requested spans are limited to 10,000 events. JSON traversal also bounds nodes, total text, and report size; exceeding a limit raises ValueError instead of returning a partially processed result.

For ASGI applications, use ASGIPromptSanitizerMiddleware with explicit protected routes and selectors such as messages[*].content and messages[*].content[*].text. With enforce=True, uninspected requests and quarantine/reject decisions are blocked by default. See integration configuration.

Policies

Backward-compatible aliases are still accepted: balanced, strict, and code.

Scope

Use this library for:

• user text before prompt interpolation
• retrieved text before it reaches the model
• tool and function argument strings
• JSON payloads with untrusted string keys or values

Do not use it as a substitute for:

• system prompt design
• tool sandboxing and allowlists
• output validation
• human review for high-risk actions

How It Fits With Guardrails And Red-Teams

llm-input-hardening is a text-integrity layer. It is designed to run before, inside, or alongside larger guardrail systems.

NeMo Guardrails exposes input/retrieval/tool rails, including jailbreak and safety flows. Promptfoo exposes red-team strategy transforms such as Base64, Hex, Homoglyph, and Emoji Smuggling. This package covers the deterministic text integrity part of that space; it should not be described as a replacement for either tool.

Useful references:

• NeMo Guardrails rails configuration
• NeMo Guardrails library integrations
• Promptfoo red-team configuration
• Promptfoo multi-input red teaming

NeMo Guardrails Mapping

NeMo Guardrails is the orchestration layer. llm-input-hardening is a deterministic text preflight layer you can call before a NeMo flow, from a custom rail, or before handing text to a downstream model/API rail.

Promptfoo Static Obfuscation Strategy Mapping

This is not a certification claim or a claim that this package passes promptfoo's full red-team suite. Treat it as a starting map for overlap with promptfoo static obfuscation strategies.

See examples/promptfoo-redteam-overlap/ for a small overlap harness that imports promptfoo's built-in static strategy transforms and classifies which generated payloads this package catches. The checked smoke fixture currently catches base64, hex, homoglyph, and emoji.

Homoglyphs And Invisible Text, Briefly

Some attacks do not rely on clever prose. They rely on text that humans and machines see differently.

The mental model:

1. Normalize text by policy.
2. Remove characters that make prompt/log/tool text ambiguous.
3. Flag suspicious shapes that should affect logging or enforcement.
4. Keep semantic safety checks separate.

CLI

llm-input-hardening sanitize --text "abc‮def"
llm-input-hardening inspect --text "abc‮def"
llm-input-hardening inspect --compact --text "abc‮def"
llm-input-hardening decide --text "abc‮def"
llm-input-hardening decide --compact --text "abc‮def"

Development

uv sync --all-extras
uv run maturin develop --release
uv run python -m pytest -q

Benchmarks And Bakeoff

The repo has two different evaluation paths.

• bench/run_benchmarks.py is a timing harness. It measures sanitizer latency and throughput on fixed synthetic inputs.
• bench/run_bakeoff.py is an evaluation harness. It runs labeled corpus cases, checks expected detections and postconditions, and computes precision, recall, benign-change rate, and stability signals.

Use the Make targets below:

make bench
make bench-plots

make bakeoff-smoke
make bakeoff-smoke-assert

make bakeoff-full
make bakeoff-full-assert
make bakeoff-diff
make bakeoff-diff-assert

What they mean:

• bakeoff-smoke is the fast local check. It evaluates only llm-input-hardening and writes results to bench/results/bakeoff_smoke_latest.json.
• bakeoff-full is the heavyweight comparison run. It installs llm-guard and confusable-homoglyphs, then writes comparable results to bench/results/bakeoff_latest.json.
• bakeoff-diff compares the current full run to a reviewed schema-4 baseline (bench/results/bakeoff_baseline_schema4.json, or BAKEOFF_BASELINE=...). Establish that baseline after reviewing a full run; historical schema-2 results are intentionally not comparable. See baseline migration.

Generated reports stay under bench/results/.

Repo Notes

• CONTRIBUTING.md covers contribution mechanics.
• SECURITY.md covers reporting and threat assumptions.
• SCOPE.md explains what the project is and is not trying to solve.