llmbelt 🧰

A tiny, zero-dependency tool belt for working with LLMs. The small utilities you end up re-writing on every project — token counting, cost estimation, retries, prompt templates, and text chunking — in one clean import.

• 🪶 Zero required dependencies — pure standard library.
• 🔌 Provider-agnostic — works with Anthropic, OpenAI, Gemini, or anything else.
• 🧪 Fully tested across Python 3.9–3.12.

Install

pip install llmbelt

# Optional: exact token counts for OpenAI-family models
pip install "llmbelt[tiktoken]"

Usage

Count tokens

from llmbelt import count_tokens, truncate_to_tokens

count_tokens("Hello, world!")              # exact if tiktoken installed, else estimated
count_tokens("Hello", model="gpt-4o")      # use a model-specific encoding

# Trim text to fit a budget (great before sending context to an API)
truncate_to_tokens(long_document, max_tokens=4000)

Estimate cost

from llmbelt import estimate_cost, Price

estimate_cost(input_tokens=1_500, output_tokens=800, model="gpt-4o-mini")   # -> USD

# Bring your own prices (the built-in table is approximate — always verify):
my_prices = {"my-model": Price(input_per_1m=2.0, output_per_1m=6.0)}
estimate_cost(1000, 500, "my-model", pricing=my_prices)

Retry with backoff

from llmbelt import retry

@retry(attempts=5, exceptions=(ConnectionError, TimeoutError))
def call_api():
    ...   # retried with exponential backoff + jitter on failure

# Works on async functions too — awaited, with non-blocking asyncio.sleep backoff
@retry(attempts=5, exceptions=(ConnectionError, TimeoutError))
async def call_api_async():
    ...

Extract JSON from a model reply

from llmbelt import extract_json

extract_json('Sure!\n```json\n{"ok": true}\n```')   # -> {"ok": True}
extract_json('The score is {"value": 0.9}.')         # -> {"value": 0.9}
extract_json("no json here", default=None)           # -> None (else raises ValueError)

Prompt templates

from llmbelt import PromptTemplate

t = PromptTemplate("Translate {text} into {language}.")
t.render(text="hello", language="French")   # "Translate hello into French."
t.render(text="hello")                      # KeyError: Missing template variables: ['language']

Fit a conversation into the context window

from llmbelt import count_message_tokens, trim_messages

messages = [
    {"role": "system", "content": "You are concise."},
    {"role": "user", "content": "..."},
    # ... a long history ...
]

count_message_tokens(messages)                      # total tokens of the chat
trim_messages(messages, max_tokens=8000)            # drop oldest turns, keep the system prompt

Cache calls so you don't pay twice

from llmbelt import cached

@cached(ttl=3600)            # remember results for an hour; unhashable args are fine
def ask(prompt: str):
    ...                      # identical prompt -> served from cache, no API call

# works on async functions too
@cached()
async def ask_async(prompt): ...

Stay under rate limits

from llmbelt import RateLimiter

limiter = RateLimiter(rate=60, per=60)   # 60 requests per minute

@limiter                                  # decorator
def call_api(): ...

with limiter:                             # or a context manager
    call_api()

Chunk text for RAG

from llmbelt import chunk_text, chunk_by_tokens, split_text

chunks = chunk_text(document, chunk_size=1000, overlap=100)
# overlapping chunks so answers aren't split across a boundary

# Budget by tokens instead of characters (exact with tiktoken installed):
chunks = chunk_by_tokens(document, chunk_size=500, overlap=50)

# Smarter: break on paragraph/sentence/word boundaries instead of mid-word
chunks = split_text(document, chunk_size=1000, overlap=100)

Track spend across calls

from llmbelt import CostTracker

tracker = CostTracker()
tracker.add(input_tokens=1_500, output_tokens=800, model="gpt-4o-mini")
tracker.add(2_000, 1_200, "gpt-4o-mini")

print(tracker)          # "2 calls, 5,500 tokens, $0.0019"
tracker.summary()       # {"calls": 2, "input_tokens": ..., "cost_usd": ...}

API reference

Development

git clone https://github.com/YoungAlpaccino/llmbelt
cd llmbelt
pip install -e ".[dev]"
pytest          # run tests
ruff check .    # lint

Publishing to PyPI (maintainer notes)

python -m build
twine upload dist/*

Before first publish: confirm the name llmbelt is free on PyPI. If taken, rename in pyproject.toml, the src/ folder, and imports (a single find-and-replace).

License

MIT — see LICENSE. Use it anywhere, including commercially.