llm-prompt-refiner 0.1.8

A lightweight Python library for optimizing and cleaning LLM inputs

Prompt Refiner

🚀 Lightweight Python library for AI Agents, RAG apps, and chatbots with smart context management and automatic token optimization. Save 10-20% on API costs while fitting RAG docs, chat history, and prompts into your token budget.

---

🎯 Perfect for:

RAG Applications • AI Agents • Chatbots • Document Processing • Cost Optimization

---

Why use Prompt Refiner?

Build AI agents, RAG applications, and chatbots with automatic token optimization and smart context management. Here's a complete example (see examples/quickstart.py for full code):

from prompt_refiner import MessagesPacker, SchemaCompressor, ResponseCompressor, StripHTML
from openai import OpenAI, pydantic_function_tool
from pydantic import BaseModel, Field

# 1. Pack messages with token budget and track savings
packer = MessagesPacker(
    max_tokens=1000,
    model="gpt-4o-mini",
    track_savings=True,
    system="You are a helpful AI assistant that helps users find books.",
    context=(
        ["<div><h1>Installation Guide</h1>...</div>"],
        [StripHTML()]
    ),
    query="Search for books about Python programming."
)
messages = packer.pack()

# Get token savings
savings = packer.get_token_savings()
print(f"Saved {savings['saved_tokens']} tokens ({savings['savings_rate']:.1%})")

# 2. Compress tool schema (139 → 131 tokens, 5.8% saved)
class SearchBooksInput(BaseModel):
    query: str = Field(description="Search query to find books")

tool_schema = pydantic_function_tool(SearchBooksInput, name="search_books")
compressed_schema = SchemaCompressor().process(tool_schema)

# 3. Execute tool call with OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=[compressed_schema]
)
tool_call = response.choices[0].message.tool_calls[0]
tool_response = search_books(**json.loads(tool_call.function.arguments))

# 4. Compress tool response (19251 → 12813 tokens, 33.4% saved)
compressed_response = ResponseCompressor().process(tool_response)

💡 Run python examples/quickstart.py to see the complete workflow with real OpenAI API verification.

Key benefits:

• Tool schema compression - Save 10-15% tokens on AI agent function definitions
• Tool response compression - Save 30-70% tokens on agent tool outputs
• Compose operations with | - Chain multiple cleaners into a pipeline
• Save 10-20% tokens - Remove HTML, whitespace, duplicates, and redact PII automatically
• Stay within budget - MessagesPacker fits everything into 1000 tokens using priority-based selection
• JIT cleaning - Clean content on-the-fly with refine_with parameter
• Production ready - Output goes directly to OpenAI without extra steps

✨ Key Features

Module	Description	Components
Cleaner	Remove noise and save tokens	StripHTML(), NormalizeWhitespace(), FixUnicode(), JsonCleaner()
Compressor	Reduce size aggressively	TruncateTokens(), Deduplicate()
Scrubber	Protect sensitive data	RedactPII()
Tools	Optimize AI agent function calling (tool schemas & responses)	SchemaCompressor(), ResponseCompressor()
Packer	Fit content within token budgets	MessagesPacker (chat APIs), TextPacker (completion APIs)
Strategy	Benchmark-tested presets for quick setup	MinimalStrategy, StandardStrategy, AggressiveStrategy

Installation

# Basic installation (lightweight, zero dependencies)
pip install llm-prompt-refiner

# With precise token counting (optional, installs tiktoken)
pip install llm-prompt-refiner[token]

Examples

Check out the examples/ folder for detailed examples:

• strategy/ - Preset strategies (Minimal, Standard, Aggressive) with benchmark results
• cleaner/ - HTML cleaning, JSON compression, whitespace normalization, Unicode fixing
• compressor/ - Smart truncation, deduplication
• scrubber/ - PII redaction (emails, phones, credit cards, etc.)
• tools/ - Tool/API output cleaning for agent systems
• packer/ - Context budget management with OpenAI integration
• analyzer/ - Token counting and cost savings tracking

📖 Full documentation: examples/README.md

📊 Proven Effectiveness

We benchmarked Prompt Refiner on 30 real-world test cases (SQuAD + RAG scenarios) to measure token reduction and response quality:

Strategy	Token Reduction	Quality (Cosine)	Judge Approval	Overall Equivalent
Minimal	4.3%	0.987	86.7%	86.7%
Standard	4.8%	0.984	90.0%	86.7%
Aggressive	15.0%	0.964	80.0%	66.7%

Key Insights:

• Aggressive strategy achieves 3x more savings (15%) vs Minimal while maintaining 96.4% quality
• Individual RAG tests showed 17-74% token savings with aggressive strategy
• Deduplicate (Standard) shows minimal gains on typical RAG contexts
• TruncateTokens (Aggressive) provides the largest cost reduction for long contexts
• Trade-off: More aggressive = more savings but slightly lower judge approval

Example: RAG with duplicates

• Minimal (HTML + Whitespace): 17% reduction
• Standard (+ Deduplicate): 31% reduction
• Aggressive (+ Truncate 150 tokens): 49% reduction 🎉

💰 Cost Savings: At scale (1M tokens/month), 15% reduction saves ~$54/month on GPT-4 input tokens.

📖 See full benchmark: benchmark/custom/README.md

⚡ Performance & Latency

"What's the latency overhead?" - Negligible. Prompt Refiner adds < 0.5ms per 1k tokens of overhead.

Strategy	@ 1k tokens	@ 10k tokens	@ 50k tokens	Overhead per 1k tokens
Minimal (HTML + Whitespace)	0.05ms	0.48ms	2.39ms	0.05ms
Standard (+ Deduplicate)	0.26ms	2.47ms	12.27ms	0.25ms
Aggressive (+ Truncate)	0.26ms	2.46ms	12.38ms	0.25ms

Key Insights:

• ⚡ Minimal strategy: Only 0.05ms per 1k tokens (faster than a network packet)
• 🎯 Standard strategy: 0.25ms per 1k tokens - adds ~2.5ms to a 10k token prompt
• 📊 Context: Network + LLM TTFT is typically 600ms+, refining adds < 0.5% overhead
• 🚀 Individual operations (HTML, whitespace) are < 0.5ms per 1k tokens

Real-world impact:

10k token RAG context refining: ~2.5ms overhead
Network latency: ~100ms
LLM Processing (TTFT): ~500ms+
Total overhead: < 0.5% of request time

🔬 Run yourself: python benchmark/latency/benchmark.py (no API keys needed)

🎮 Interactive Demo

Try prompt-refiner in your browser - no installation required!

🚀 Launch Interactive Demo →

Play with different strategies, see real-time token savings, and find the perfect configuration for your use case. Features:

• 🎯 6 preset examples (e-commerce, support tickets, docs, RAG, etc.)
• ⚡ Quick strategy presets (Minimal, Standard, Aggressive)
• 💰 Real-time cost savings calculator
• 🔧 All 7 operations configurable
• 📊 Visual metrics dashboard

Star History

License

MIT