llmslim 0.2.0

from llmslim import compress

result = compress(your_massive_prompt, target_ratio=0.5)
# That's it. 50% fewer tokens. Same meaning. Half the cost. 🚀

You are an AI assistant that helps users with their
coding questions. You should be helpful, harmless,
and honest. When answering questions, you should
provide detailed explanations with code examples
where appropriate. Make sure to consider edge cases
and provide best practices. If you're not sure about
something, say so rather than making things up.
Please format your responses using markdown for
better readability. Include relevant links to
documentation when possible. Always test your code
before sharing it. Remember to handle errors
gracefully and explain your reasoning step by step...

[... 200 more lines of context ...]

You are an AI assistant for coding questions.
Be helpful, harmless, honest. Provide detailed
explanations with code examples. Consider edge
cases and best practices. If unsure, say so.
Format responses in markdown. Include documentation
links. Always test code before sharing. Handle
errors gracefully, explain reasoning step by step.

[... compressed with meaning preserved ...]

🎯 Why llmslim?

⚡ Quickstart

Installation

# Core (works offline, no model downloads needed)
pip install llmslim

# With high-quality semantic embeddings (recommended)
pip install "llmslim[semantic]"

# Everything (semantic + fast token counting + NLTK sentence splitting)
pip install "llmslim[all]"

One Line Is All You Need

from llmslim import compress

result = compress(your_prompt, target_ratio=0.5)

print(result.compressed_text)      # → your compressed prompt
print(result.reduction_percent)    # → 52.3
print(result.tokens_saved)         # → 1,847
print(result.summary())            # → full stats breakdown

Use Directly With Any LLM

from llmslim import compress
from openai import OpenAI

client = OpenAI()

# Compress before sending — drop-in, zero friction
prompt = compress(massive_system_prompt, target_ratio=0.5)

response = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": str(prompt)},  # ← compressed!
        {"role": "user", "content": user_question},
    ],
)
# Same quality response. Half the cost.

🧠 How It Works

graph LR
    A["📝 Input Text<br/><i>3,000 tokens</i>"] --> B["✂️ Sentence<br/>Splitting"]
    B --> C["🧩 Semantic<br/>Chunking"]
    C --> D["📊 Extractive<br/>Ranking"]
    D --> E["🔒 Instruction<br/>Preservation"]
    E --> F["🎯 Budget-Aware<br/>Selection"]
    F --> G["✨ Output<br/><i>1,500 tokens</i>"]

    style A fill:#1a1b27,stroke:#58a6ff,color:#c9d1d9
    style B fill:#1a1b27,stroke:#7c3aed,color:#c9d1d9
    style C fill:#1a1b27,stroke:#7c3aed,color:#c9d1d9
    style D fill:#1a1b27,stroke:#f778ba,color:#c9d1d9
    style E fill:#1a1b27,stroke:#ffa657,color:#c9d1d9
    style F fill:#1a1b27,stroke:#ffa657,color:#c9d1d9
    style G fill:#1a1b27,stroke:#10b981,color:#c9d1d9

The 6-Step Pipeline

🔥 Features

🤝 Works With Every LLM

llmslim is model-agnostic. It compresses the text before it reaches any model. Works with any API, any framework, any model.

💬 Compress Chat Histories

from llmslim import compress_chat_messages

conversation = [
    {"role": "system", "content": "You are a helpful coding assistant..."},
    {"role": "user", "content": very_long_user_message},
    {"role": "assistant", "content": very_long_assistant_response},
    {"role": "user", "content": follow_up_question},
]

# Compress user & assistant messages, preserve system prompt
compressed = compress_chat_messages(conversation, target_ratio=0.5)

# Use directly with OpenAI, Anthropic, etc.
response = client.chat.completions.create(model="gpt-5", messages=compressed)

📚 RAG Pipeline Compression

from llmslim import compress_documents

# Your retrieved chunks from a vector DB
retrieved_chunks = [chunk1, chunk2, chunk3, chunk4, chunk5]
user_query = "How do I handle authentication in FastAPI?"

# Query-aware compression: keeps sentences relevant to the question
results = compress_documents(
    retrieved_chunks,
    query=user_query,
    target_ratio=0.4,  # aggressive 60% reduction
)

# Build compressed context
context = "\n\n".join(r.compressed_text for r in results)
total_saved = sum(r.tokens_saved for r in results)
print(f"Saved {total_saved} tokens across {len(results)} documents")

💰 Cost Savings Calculator

from llmslim import compress, estimate_cost_savings

result = compress(prompt, target_ratio=0.5)

savings = estimate_cost_savings(
    original_tokens=result.original_tokens,
    compressed_tokens=result.compressed_tokens,
    model="gpt-5",
    requests_per_day=50_000,
)

print(savings.summary())

Model: gpt-5 ($0.00125/1K input tokens)
Tokens saved per request: 1,423 (51.2%)
At 50,000 requests/day:
  Daily savings:   $88.94
  Monthly savings: $2,668.13
  Annual savings:  $32,462.19

📊 Performance & Benchmarks

llmslim undergoes continuous benchmark evaluation across multi-domain prompt corpora. Below are visual representations of system performance, instruction preservation, and memory efficiency.

🖼️ Performance Graphs

Processing Latency	Token Reduction Efficiency

Information Retention Metrics	Memory Footprint Allocation

📋 Empirical Benchmark Quality Results

Compression quality evaluated across standardized benchmark corpora (datasets/):

📌 Key Scientific Finding: Explicit instruction directives (sentences with "must", "never", "ensure", code block fences, markdown titles) are preserved at 100.0% across all compression ratios.

🔬 Methodology & Hardware Specifications

• Evaluation Hardware: x86_64 CPU @ 3.4 GHz, 16 GB System Memory.
• Runtime Environment: Python 3.12.5, Windows 11 / Ubuntu 22.04 LTS.
• Dataset Reference: Benchmark texts sourced from datasets/arxiv_summarization.json, synthetic multi-turn dialogs, and technical API documentations.
• Reproducibility: All charts and metrics are generated automatically via:

  # Execute reproducible regression benchmark
  python benchmarks/benchmark_regression.py
  
  # Generate vector chart visualizers
  python assets/generate_benchmark_charts.py
  

🗺️ Product Roadmap

• v0.1.0 — Foundation Release
  • Core TF-IDF sentence centrality ranking.
  • Basic sentence boundary splitting and token counter.
• v0.2.0 — Production & Governance Release (Current)
  • 🔒 Instruction Retention Engine: 100% preservation of System Prompts, code blocks, and imperative directives.
  • 🏷️ Entity & Pattern Preservation: Named entities, dates, URLs, numbers, and custom regex pattern protection.
  • 🧩 Semantic Chunking: Topic drift detection and chunk-proportional sentence selection.
  • 🛠️ Production Quality Suite: 93%+ test coverage, Ruff/Black/MyPy compliance, and multi-OS CI matrix.
• v0.3.0 — High-Throughput Engine (In Progress)
  • 🌊 Streaming Compression API (compress_stream): Compresses token streams on-the-fly.
  • ⚡ Native ONNX Embeddings: Sub-5ms deep semantic embeddings without heavy PyTorch dependency.
  • 🔄 Async Pipeline Integration: Native asyncio batch helper (acompress_batch).
• v1.0.0 — Ecosystem Maturity (Future)
  • 🌐 WASM / Web Assembly Build: Run prompt compression locally inside browser & Edge runtimes.
  • 🖼️ Multi-Modal Context Compression: Token compression for vision-language models & interleaved text-image prompts.
  • ⚡ C-Extension Core Acceleration: Ultra-fast sentence tokenizer written in Rust/C (<1ms overhead per 10k tokens).

🛠️ Advanced Configuration

from llmslim import ContextCompressor

compressor = ContextCompressor(
    # Chunking parameters
    max_chunk_tokens=180,          # soft cap per semantic chunk
    similarity_threshold=0.35,     # topic drift sensitivity (lower = larger chunks)

    # Compression behavior
    min_tokens_for_compression=40, # skip tiny texts

    # Scoring weights (tune to your use case)
    weights={
        "centrality": 0.35,        # how representative of the chunk
        "position": 0.15,          # first/last sentence bonus
        "entity": 0.15,            # named entities, numbers, URLs
        "instruction": 0.25,       # directive language boost
        "query": 0.35,             # query relevance (RAG mode)
        "length_penalty": 0.20,    # penalize very short sentences
    },

    # Custom preservation rules
    preserve_patterns=[
        r"API_KEY",                # always keep sentences mentioning API keys
        r"^WARNING:",              # keep warning lines
        r"https?://",              # keep sentences with URLs
    ],
)

result = compressor.compress(text, target_ratio=0.5, query="optional query")

🖥️ CLI Usage

# Basic compression
llmslim input.txt -r 0.5 -o compressed.txt

# With stats
llmslim input.txt --ratio 0.5 --stats

# With cost estimate
llmslim input.txt -r 0.5 --cost gpt-5 --requests-per-day 10000

# From stdin
cat prompt.txt | llmslim --ratio 0.4

# Pipe to clipboard (macOS)
llmslim input.txt -r 0.5 | pbcopy

📦 API Reference

🏗️ Architecture

llmslim/
├── __init__.py          # Public API exports
├── core.py              # ContextCompressor class + compress() function
├── chunking.py          # Semantic chunking with topic-drift detection
├── ranking.py           # Multi-signal sentence scoring (centrality, entities, instructions)
├── embeddings.py        # Pluggable backends: sentence-transformers + TF-IDF fallback
├── tokenization.py      # Sentence/paragraph splitting with code-block protection
├── tokens.py            # Token counting (tiktoken with heuristic fallback)
├── cost.py              # Cost savings estimation for popular LLM models
├── pipelines.py         # High-level helpers: chat compression, document batches
└── cli.py               # Command-line interface

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

# Development setup
git clone https://github.com/Thanatos9404/llmslim.git
cd llmslim
pip install -e ".[all,dev]"
pytest tests/ -v

📄 License

MIT License — see LICENSE for details.

⭐ Star History

If this project saved you money, star it! ⭐

Built with ❤️ by Yashvardhan Thanvi