Prompt Refiner

🚀 Lightweight Python library for AI Agents, RAG apps, and chatbots with smart context management and automatic token optimization. Save 5-70% on API costs - 57% average reduction on function calling, 5-15% on RAG contexts.

🎯 Perfect for:

RAG Applications • AI Agents • Chatbots • Document Processing • Cost Optimization

Why use Prompt Refiner?

Build AI agents, RAG applications, and chatbots with automatic token optimization and smart context management. Here's a complete example (see examples/quickstart.py for full code):

from prompt_refiner import MessagesPacker, SchemaCompressor, ResponseCompressor, StripHTML, NormalizeWhitespace

# 1. Pack messages (automatic refining with default strategies)
packer = MessagesPacker(
    track_tokens=True,
    system="<p>You are a helpful AI assistant.</p>",
    context=(["<div>Installation Guide...</div>"], StripHTML() | NormalizeWhitespace()),
    query="<span>Search for Python books.</span>"
)
messages = packer.pack()

# 2. Compress tool schema
tool_schema = pydantic_function_tool(SearchBooksInput, name="search_books")
compressed_schema = SchemaCompressor().process(tool_schema)

# 3. Call LLM with compressed schema
response = client.chat.completions.create(
    model="gpt-4o-mini", messages=messages, tools=[compressed_schema]
)

# 4. Compress tool response
tool_response = search_books(**json.loads(tool_call.function.arguments))
compressed_response = ResponseCompressor().process(tool_response)

Default refining strategies:

• system/query: MinimalStrategy (StripHTML + NormalizeWhitespace)
• context/history: StandardStrategy (StripHTML + NormalizeWhitespace + Deduplicate)
• Override with tuple: context=(docs, StripHTML() | NormalizeWhitespace())

💡 Run python examples/quickstart.py to see the complete workflow with real OpenAI API verification.

Key benefits:

• Default strategies - Automatic refining (MinimalStrategy for system/query, StandardStrategy for context/history)
• Tool schema compression - Save 10-70% tokens on AI agent function definitions (avg: 57%)
• Tool response compression - Save 30-70% tokens on agent tool outputs
• Compose operations with | - Chain multiple cleaners into a pipeline
• Save 5-15% tokens on RAG contexts - Remove HTML, whitespace, duplicates automatically
• All items included - No token budget limits, let LLM APIs handle final truncation
• Track savings - Measure token optimization impact with built-in savings tracking
• Production ready - Output goes directly to OpenAI without extra steps

✨ Key Features

Installation

# Basic installation (lightweight, zero dependencies)
pip install llm-prompt-refiner

# With precise token counting (optional, installs tiktoken)
pip install llm-prompt-refiner[token]

Examples

Check out the examples/ folder for detailed examples:

• strategy/ - Preset strategies (Minimal, Standard, Aggressive) with benchmark results
• cleaner/ - HTML cleaning, JSON compression, whitespace normalization, Unicode fixing
• compressor/ - Smart truncation, deduplication
• scrubber/ - PII redaction (emails, phones, credit cards, etc.)
• tools/ - Tool/API output cleaning for agent systems
• packer/ - Context budget management with OpenAI integration
• analyzer/ - Token counting and cost savings tracking

📖 Full documentation: examples/README.md

📊 Proven Effectiveness

Prompt Refiner has been rigorously tested across 3 comprehensive benchmark suites covering function calling, RAG applications, and performance. Here's what the data shows:

🎯 Function Calling Benchmark: 57% Average Token Reduction

SchemaCompressor was tested on 20 real-world API schemas from Stripe, Salesforce, HubSpot, Slack, OpenAI, Anthropic, and more:

Key Highlights:

• ✨ 56.9% average reduction across all schemas (15,342 tokens saved)
• 🔒 100% lossless compression - all protocol fields preserved (name, type, required, enum)
• ✅ 100% callable (20/20 validated) - all compressed schemas work correctly with OpenAI function calling
• 🏢 Enterprise APIs see 70%+ reduction - HubSpot, Salesforce, OpenAI File Search
• 📊 Real-world schemas from production APIs, not synthetic examples
• ⚡ Zero API cost - local processing with tiktoken

✅ Functional Validation:

We tested all 20 compressed schemas with real OpenAI function calling to prove they work correctly:

• 100% callable (20/20): Every compressed schema successfully triggers function calls
• 60% identical (12/20): Majority produce exactly the same arguments as original schemas
• 40% different but valid (8/20): Compressed descriptions may influence LLM's choice among valid options (e.g., default values, placeholders)
• Bottom line: Compression is safe for production - schemas remain functionally correct

💰 Cost Savings Example: A medium agent (10 tools, 500 calls/day) saves $541/month with SchemaCompressor.

📖 See full benchmark: benchmark/README.md#function-calling-benchmark

📚 RAG & Text Optimization Benchmark: 5-15% Token Reduction

Tested on 30 real-world test cases (SQuAD + RAG scenarios) to measure token reduction and quality preservation:

Key Insights:

• ✅ Standard strategy: 5% reduction with 98.4% cosine similarity and 90% judge approval
• 🚀 Aggressive strategy: 15% reduction while maintaining 96.4% semantic quality
• 📊 Individual tests: Up to 74% token savings on contexts with HTML and duplicates

💰 Cost Savings: At 1M tokens/month, 15% reduction saves $54/month on GPT-4 input tokens.

📖 See full benchmark: benchmark/README.md#rag-quality-benchmark

⚡ Performance & Latency

"What's the latency overhead?" - Negligible. Prompt Refiner adds < 0.5ms per 1k tokens of overhead.

Key Insights:

• ⚡ Minimal strategy: Only 0.05ms per 1k tokens (faster than a network packet)
• 🎯 Standard strategy: 0.25ms per 1k tokens - adds ~2.5ms to a 10k token prompt
• 📊 Context: Network + LLM TTFT is typically 600ms+, refining adds < 0.5% overhead
• 🚀 Individual operations (HTML, whitespace) are < 0.5ms per 1k tokens

Real-world impact:

10k token RAG context refining: ~2.5ms overhead
Network latency: ~100ms
LLM Processing (TTFT): ~500ms+
Total overhead: < 0.5% of request time

🔬 Run yourself: python benchmark/latency/benchmark.py (no API keys needed)

🎮 Interactive Demo

Try prompt-refiner in your browser - no installation required!

Play with different strategies, see real-time token savings, and find the perfect configuration for your use case. Features:

• 🎯 6 preset examples (e-commerce, support tickets, docs, RAG, etc.)
• ⚡ Quick strategy presets (Minimal, Standard, Aggressive)
• 💰 Real-time cost savings calculator
• 🔧 All 7 operations configurable
• 📊 Visual metrics dashboard

Star History

License

MIT