llm-input-hardening 1.2.0

Deterministic Unicode-aware input hardening for LLM applications.

llm-input-hardening

Deterministic Unicode-aware input hardening for LLM applications.

llm-input-hardening sits between untrusted text and your prompt or template layer. It normalizes Unicode, removes invisible or control characters that destabilize prompts, emits structured reports for logging and enforcement, and can add confusable mixed-script signals when you want stricter checks.

It is not prompt injection prevention. It addresses text-integrity problems, not model intent.

This repository keeps docs intentionally lightweight. The README is the main reference.

Install

pip install llm-input-hardening

Optional extras:

• pip install "llm-input-hardening[security]" for the confusable_homoglyphs backend
• pip install "llm-input-hardening[ftfy]" for pre-sanitize text repair

Import from llm_input_hardening.

Quickstart

from llm_input_hardening import sanitize

clean_text, report = sanitize(user_text, policy="balanced_chat")

Recursive JSON sanitization:

from llm_input_hardening import sanitize_json

clean_payload, report = sanitize_json(payload, policy="balanced_chat")

Enforcement-oriented usage:

from llm_input_hardening import sanitize_and_decide

clean_text, report, decision = sanitize_and_decide(
    user_text,
    sanitize_policy="strict_exec",
    confusables_backend="confusable_homoglyphs",
)

Policies

Policy	Normalization	Default-Ignorables	Whitespace tidy	Typical use
preserve	NFC	preserve	no	low-friction logging and observability
balanced_chat	NFC	preserve	yes	general chat and prompt input
strict_exec	NFKC	strip	yes	tool arguments and execution-adjacent paths
code_mode	NFC	strip	no	code snippets and syntax-sensitive text

Backward-compatible aliases are still accepted: balanced, strict, and code.

Scope

Use this library for:

• user text before prompt interpolation
• retrieved text before it reaches the model
• tool and function argument strings
• JSON payloads that contain prompt-bearing string leaves

Do not use it as a substitute for:

• system prompt design
• tool sandboxing and allowlists
• output validation
• human review for high-risk actions

CLI

llm-input-hardening sanitize --text "abc‮def"
llm-input-hardening inspect --text "abc‮def"
llm-input-hardening decide --text "abc‮def"

Development

uv sync --all-extras
uv run maturin develop --release
uv run python -m pytest -q

Benchmarks And Bakeoff

The repo has two different evaluation paths.

• bench/run_benchmarks.py is a timing harness. It measures sanitizer latency and throughput on fixed synthetic inputs.
• bench/run_bakeoff.py is an evaluation harness. It runs labeled corpus cases, checks expected detections and postconditions, and computes precision, recall, benign-change rate, and stability signals.

Use the Make targets below:

make bench
make bench-plots

make bakeoff-smoke
make bakeoff-smoke-assert

make bakeoff-full
make bakeoff-full-assert
make bakeoff-diff
make bakeoff-diff-assert

What they mean:

• bakeoff-smoke is the fast local check. It evaluates only llm-input-hardening and writes results to bench/results/bakeoff_smoke_latest.json.
• bakeoff-full is the heavyweight comparison run. It installs llm-guard and confusable-homoglyphs, then writes comparable results to bench/results/bakeoff_latest.json.
• bakeoff-diff compares the current full run to bench/results/bakeoff_baseline_v1.0.0.json.

Generated reports stay under bench/results/.

Repo Notes

• CONTRIBUTING.md covers contribution mechanics.
• SECURITY.md covers reporting and threat assumptions.
• SCOPE.md explains what the project is and is not trying to solve.