token-sentinel

Predictive token-waste detection for AI agents — a Python SDK that runs in-process, watches LLM calls after each provider response, and fires a typed callback so you can log, alert, or hard-stop the next turn.

Observability tools show the bill after the fact. TokenSentinel names the waste pattern while the session is still running. Detection is post-call (the turn that just finished is already billed); intervention saves subsequent turns.

Docs: https://docs.tokensentinel.dev

Install

pip install token-sentinel[anthropic]   # or [openai], [gemini], …
# optional: local token estimates when usage is missing
pip install token-sentinel[tiktoken]

Quick start

from token_sentinel import Sentinel
import anthropic

sentinel = Sentinel(project="my-agent", mode="log")  # log | alert | block

@sentinel.on_waste  # or @sentinel.on_leak — same callback
def handle(event):
    print(f"{event.type} conf={event.confidence:.2f} burn≈${event.estimated_burn:.4f}")

client = sentinel.wrap(anthropic.Anthropic())
client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=100,
    messages=[{"role": "user", "content": "Hello"}],
)

Tune rule thresholds

Every rule exposes flat config keys "rule_name.param". Example — require five similar tool calls (default is 3) before tool_loop fires:

sentinel = Sentinel(
    project="my-agent",
    mode="log",
    config={
        "tool_loop.min_calls": 5,
        "tool_loop.cosine_threshold": 0.80,
        "tool_loop.window_seconds": 120,
    },
)

Full parameter tables: docs/user/04-waste-rules.md.

What it catches (15 rules)

Providers

Native wrappers (pip install token-sentinel[<extra>]):

OpenAI-compatible hosts (same wrapper, set base_url): DeepSeek, Together, Fireworks, Groq, OpenRouter, Mistral, Perplexity, vLLM, Ollama, TGI, LM Studio, xAI Grok (https://api.x.ai/v1), etc.

import openai
client = sentinel.wrap(
    openai.OpenAI(api_key="…", base_url="https://api.deepseek.com")
)

Pass a stable _sentinel_session_id=... on each call (or use Sentinel.session(...)) so multi-turn rules see history.

Modes

Cost estimates (estimated_burn)

Burn figures are approximate FinOps signals, not invoices:

• Model-aware rates for major Anthropic / OpenAI / Gemini / DeepSeek / Cohere / Mistral families (input vs output priced separately).
• Prompt-cache reads (OpenAI cached_tokens, Anthropic cache_read_input_tokens) discounted when present on the usage payload.
• Unknown models fall back to a flat average (~$9e-6 / token).
• Optional tiktoken fills missing token counts when usage was omitted.

Override rates with Sentinel(pricing_table={...}) or from token_sentinel import estimate_usd, ModelRate, default_pricing_table.

Frameworks

Instruments at the LLM-client layer, so MCP hosts, RAG pipelines, LangChain / LangGraph / CrewAI / AutoGen / Pydantic AI work when traffic bottoms out in a wrapped client. Enrichers: pip install token-sentinel[langchain] or [otel].

Optional cloud

Nothing phones home unless you pass both cloud_endpoint= and api_key= on Sentinel(...). For hosted dashboard, Composite Signals and policy features — see tokensentinel.dev. This package is Apache-2.0 and fully usable offline.

Status

1.0.3 — 15 rules, model-aware burn estimates, cache-aware usage, optional tiktoken fallback, OSS-focused docs.

Public API (Sentinel, wrap, on_leak / on_waste, record_call, session, mark_long_running, events/exceptions, pricing helpers) follows semver — pin deliberately (token-sentinel>=1.0,<2).

More docs

• User guide — install, modes, rules, providers, API reference
• Architecture · Waste taxonomy
• CHANGELOG

Contact

support@tokensentinel.dev · hello@tokensentinel.dev · tokensentinel.dev

License

Apache-2.0 — see LICENSE.