promptbudget

Deterministic context-window budgeting for LLM prompts — with a full audit trail of every token decision.

Your system prompt, tool schemas, chat history and retrieved chunks are all competing for the same context window. Most codebases resolve that fight with a while total > limit: messages.pop(0) loop buried in a helper — which silently evicts the one document the answer depended on, and leaves you no way to find out.

promptbudget makes that allocation an explicit, testable, auditable decision.

from promptbudget import Section, Item, pack

result = pack(
    [
        Section("system", SYSTEM_PROMPT, priority=100, strategy="all_or_nothing"),
        Section("tools", tool_schemas, priority=90, min_tokens=2_000),
        Section("history", messages, priority=50, strategy="drop_oldest"),
        Section("retrieval", chunks, priority=40, strategy="lowest_score_first"),
        Section("scratchpad", notes, priority=10, max_tokens=1_000),
    ],
    budget=128_000,
    reserved_output=4_000,
)

send(result.text)

for d in result.dropped():
    log.info("%s[%d] %s: %s", d.section, d.index, d.action.value, d.reason)

retrieval[7] dropped: lowest relevance score, evicted first to fit the section allocation
scratchpad[0] truncated: cut to the tokens remaining in the section allocation

Why this exists

Existing tools each cover a slice: LangChain's trim_messages trims message lists, LLMLingua compresses text, tiktoken counts tokens. None of them arbitrate a fixed budget across several sections with declared guarantees and tell you what they did.

Install

pip install promptbudget                 # zero dependencies
pip install "promptbudget[tiktoken]"     # exact OpenAI-family token counts

Python 3.9+. The core has no runtime dependencies.

Core ideas

Sections compete; priority decides who wins

Sections are served highest priority first. Ties break on name, so identical inputs always produce an identical prompt — the same reason you pin dependency versions.

Rendering follows your list order, not priority order. Priority decides who gets tokens; you decide what the prompt reads like.

Guarantees are kept or the call fails

Section("tools", tool_schemas, priority=90, min_tokens=2_000)

min_tokens is reserved before any lower-priority section is served — a greedy high-priority section cannot eat tokens promised downstream. If the minimums collectively don't fit, you get BudgetTooSmallError rather than a prompt that quietly violates one:

BudgetTooSmallError: Guaranteed minimums require 6000 tokens but only 4000 are
available (budget=8000, reserved_output=4000). Lower a min_tokens value, raise
the budget, or reduce reserved_output.

max_tokens is the mirror image: a cap, so a runaway history can't crowd out retrieval even when the window is empty.

Reserve room for the reply

pack(sections, budget=128_000, reserved_output=4_000)

Forgetting the model's own output is the most common way to get a context-length error from a prompt that "obviously fits". Reserved tokens are never offered to a section.

Strategies decide what gets shed

Pin anything that must survive:

Item("The user's actual question", pinned=True)

Pinned items are never dropped or cut. If a pinned item genuinely cannot fit, you get SectionOverflowError — a loud failure beats a mangled prompt.

Bring your own by subclassing Strategy:

from promptbudget import Strategy, StrategyResult, register_strategy


class DeduplicateThenTruncate(Strategy):
    name = "dedupe"

    def apply(
        self, section, items, allocated, tokenizer, separator
    ): ...  # return StrategyResult(items, decisions)


register_strategy("dedupe", DeduplicateThenTruncate())

Any tokenizer, or none

from promptbudget import TiktokenTokenizer, pack

pack(sections, budget=128_000, tokenizer=TiktokenTokenizer("gpt-4o"))
pack(sections, budget=8_000, tokenizer=lambda text: len(text) // 4)  # any str -> int
pack(sections, budget=8_000)  # built-in heuristic

The allocator never imports a tokenizer directly — it talks to the Tokenizer interface, so HuggingFace, a provider SDK, or your own estimator all drop straight in.

The result explains itself

result.text  # the fitted prompt
result.used_tokens  # what it actually costs
result.remaining_tokens  # headroom left
result.complete  # True when nothing was lost
result.dropped()  # every item that lost content, and why
result.section("rag")  # per-section accounting
result.to_json(indent=2)  # structured logging, ready to ship

Because packing is a pure function returning a plain value, you can assert on it:

def test_the_user_question_always_survives():
    result = pack(sections, budget=4_000)
    assert "user_question" not in {d.section for d in result.dropped()}

Guarantees

Verified by property-based tests across hundreds of generated inputs (tests/test_properties.py):

• The output never exceeds the budget.
• Reserved output tokens are never consumed.
• max_tokens caps are never breached.
• Identical inputs produce byte-identical prompts.
• Every item receives exactly one decision — nothing vanishes unrecorded.
• complete never lies about dropped content.

Development

git clone https://github.com/mathewOracle/promptbudget && cd promptbudget
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pytest && ruff check . && mypy

Contributions welcome — see CONTRIBUTING.md. New strategies are especially welcome, and shouldn't require touching the allocator at all.

See CHANGELOG.md for release history and RELEASING.md for how versions get published.

License

MIT — see LICENSE.