context-clipper ✂️

context-clipper is a lightweight, framework-agnostic Python library designed to solve LLM context window overflow and "lost in the middle" retrieval degradation.

Unlike naive sliding windows (messages[-10:]) that blindly drop critical system prompts or early user context, context-clipper treats the conversation window as a prioritized memory heap to intelligently shed low-priority noise or summarize intermediate turns without pulling in heavy frameworks like LangChain, LlamaIndex, or Pydantic.

✨ Why context-clipper?

• 🪶 Zero Bloat: Built using 100% pure Python standard libraries (dataclasses, heapq, functools). No heavy framework dependencies.
• 🎯 Accurate Token Counting: Uses tiktoken when available (including ~4 tokens/message framing overhead) with graceful fallback to character heuristics.
• 🔌 Standard API Compatibility: Accepts and returns standard Python lists of dictionaries matching OpenAI and Anthropic message formats.
• 🧠 3-Phase Waterfall Engine:
  1. Total Protection: Guarantees all sticky=True messages (e.g., system instructions, core rules) are preserved.
  2. Priority-Based Shedding: Uses a min-heap to drop lowest-priority items first (e.g., failed web searches or debug logs with priority=0).
  3. Rolling Middle-Summarization: Condenses intermediate conversational turns into a single summary message via your own custom LLM callback while preserving active recent tail turns.

📦 Installation

Install directly via pip:

pip install context-clipper

To include tiktoken for tokenizer-accurate counting:

pip install context-clipper[tiktoken]

🚀 Quickstart

Basic Priority Shedding

from context_clipper import pack

messages = [
    # System instructions are sticky and cannot be dropped
    {"role": "system", "content": "You are a helpful coding assistant.", "sticky": True, "priority": 100},
    {"role": "user", "content": "How do I reverse a list in Python?", "priority": 1},
    {"role": "assistant", "content": "You can use list.reverse() or [::-1].", "priority": 1},
    # Low-priority tool error / noise that should be dropped first when budget is tight
    {"role": "tool", "content": "Error 500: Search timeout", "priority": 0, "tool_call_id": "call_1"},
    {"role": "user", "content": "Thanks! Now explain generator expressions.", "priority": 2}
]

# Pack the conversation into a 150-token window
packed_messages = pack(messages, max_tokens=150)

Rolling Middle-Summarization with Custom Callback

Pass any summarizer callback (OpenAI, Anthropic, local Ollama, etc.) to condense intermediate turns while keeping sticky prompts and recent turns intact:

from context_clipper import Clipper

def my_llm_summarizer(text: str) -> str:
    # Call your preferred LLM provider here
    return "User asked about list reversal; assistant provided reverse() and slice methods."

clipper = Clipper(max_tokens=150, summarizer_cb=my_llm_summarizer)
packed_messages = clipper.pack(messages)

print(packed_messages[1])
# {
#   "role": "system",
#   "content": "[Summary of older conversation: User asked about list reversal; assistant provided reverse() and slice methods.]"
# }

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.