chronicle-gist 0.2.2

The State Sidecar for AI Agents

📖 Chronicle-Gist: The State Sidecar for AI Agents

Chronicle-Gist is a lightweight Python library designed to solve the "Context Window Bloat" in agentic workflows. It distills massive conversation histories into a permanent, structured State Ledger, allowing your agents to remember everything while paying for almost nothing.

---

🚀 The Problem: The Loyalty Tax

The more a user talks to your agent, the more expensive that user becomes. By turn 50, you are paying for 5,000+ tokens of redundant history just to get a simple answer.

Chronicle-Gist reduces this by 80%+ using a tiered memory approach.

---

✨ Features

• Tiered Memory: Combines a high-resolution buffer (L1) with a permanent narrative summary (L2).
• Fact Ledger: Automatically extracts and persists a JSON "Source of Truth" (e.g., user preferences, names).
• Arbitrage Logic: Uses cheap models (Llama 8B) to compress history for expensive models (GPT-4o/Llama 70B).
• Zero-Dementia: Ensures critical facts are never "summarized away" during compression.

---

📦 Installation

pip install chronicle-gist

---

🛠️ Quick Start

Chronicle-Gist is designed as a Sidecar. You call it before you call your LLM to prepare the perfect, lean prompt.

import os
import asyncio
from chronicle_gist import Chronicle

# 1. Initialize with your API Key (Groq, OpenAI, etc.)
# "Arbitrage": Use a cheap model (e.g., Llama 3 8B) for the compression work.
chronicle = Chronicle(
    api_key=os.getenv("GROQ_API_KEY"), 
    model_name="groq/llama-3.1-8b-instant",
    token_threshold=1000
)

async def main():
    session_id = "user_123"

    # 2. Your Agent's State
    history = [
        {"role": "user", "content": "My name is Alex and I'm allergic to peanuts."},
        {"role": "assistant", "content": "Noted, Alex."},
        # ... imagine 50 more messages ...
    ]
    
    new_msg = {"role": "user", "content": "Can I eat this Pad Thai?"}

    # 3. The Sidecar hydrates the state and returns a lean prompt
    # It extracts facts (Allergy: Peanuts) and compresses the rest.
    context = await chronicle.process_async(session_id, new_msg, history)

    # 4. Call your LLM as usual with the optimized prompt
    # (Example using generic print, but you'd pass `context['hydrated_messages']` to OpenAI/Anthropic)
    print("--- Optimized Prompt ---")
    print(context['hydrated_messages'][0]['content'])
    # Output:
    # [MEMORY RECALL]
    # Summary: User Alex has discussed food preferences...
    # [FACT LEDGER]
    # { "name": "Alex", "allergies": ["Peanuts"] }

    print(f"\nTokens Saved: {context['meta']['tokens_saved']}")

if __name__ == "__main__":
    asyncio.run(main())

---

📊 The "Arbitrage" Dashboard

When you use Chronicle-Gist, you get real-time transparency into your agent's cognitive efficiency.

Metric	Naive Approach	Chronicle-Gist
Prompt Size	5,200 tokens	410 tokens
Latency	1.8s	0.4s
Cost	$0.0050	$0.0004

---

🏗️ Architecture

Chronicle-Gist follows the Sawtooth Memory Pattern. Every time your history hits the limit, it's compressed into the permanent state, resetting the context window while preserving the "Essence."

---

🤝 Contributing

We love contributors! Whether it's fixing a bug or adding a new "State Provider" (like Redis or Cassandra), feel free to open a PR.

---

🛡️ License

MIT License.