Skip to main content

Pack context chunks into a budget by relevance and priority. Python port of @mukundakatta/context-window-packer.

Project description

context-window-packer-py

PyPI Python License: MIT

Pack context chunks into a budget by relevance and priority. Sort by score (descending), break ties by priority, accept each chunk if it fits, otherwise drop. Zero runtime dependencies.

Python port of @mukundakatta/context-window-packer.

Install

pip install context-window-packer-py

Quick start

from context_window_packer import pack

chunks = [
    {"text": "Pluto was reclassified...", "priority": 1, "relevance": 0.9},
    {"text": "Mars is the fourth planet...", "priority": 0, "relevance": 0.4},
    {"text": "Cats are independent...", "priority": 0, "relevance": 0.1},
]

result = pack(chunks, budget_tokens=20)

result.chunks   # list of dicts that fit
result.tokens   # int -- tokens consumed by kept chunks
result.dropped  # int -- count of chunks not kept

API

pack(chunks, budget_tokens, model=None) -> PackResult

Each chunk is a mapping with at minimum text. Optional fields:

Field Default Meaning
text "" The chunk content.
tokens ceil(len(text)/4) Pre-counted tokens (skips estimator).
relevance / score 0 Higher = more relevant; primary sort key (descending).
priority 0 Tie-breaker for relevance (descending).

PackResult is a dataclass with:

  • chunks: list[dict] -- chunks that fit, in pack order.
  • tokens: int -- total tokens consumed.
  • dropped: int -- count of chunks dropped.

model is reserved for future per-family token estimators (currently uses the standard ceil(chars/4) heuristic, matching the JS sibling).

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

context_window_packer_py-0.1.0.tar.gz (4.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

context_window_packer_py-0.1.0-py3-none-any.whl (5.0 kB view details)

Uploaded Python 3

File details

Details for the file context_window_packer_py-0.1.0.tar.gz.

File metadata

File hashes

Hashes for context_window_packer_py-0.1.0.tar.gz
Algorithm Hash digest
SHA256 db83737b7303dad40b7a64c689034da27ef4eccaa83ed27df21fee2ed627f6f4
MD5 924f5d2c352839da07584cddb9ccf7c2
BLAKE2b-256 f809b3c539cc32caccb90fdf3c3808a81bac19af5cb54deef11ff5b66a393a12

See more details on using hashes here.

File details

Details for the file context_window_packer_py-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for context_window_packer_py-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4b7a0b39e69ceb73182ba86f08bc9a531fedd07af9a83ab47ed6b69f258760a2
MD5 fe56d47e0a73781a7f8224a0be68e105
BLAKE2b-256 7d0eb0c395daf44532203f40ffce46850472c0986432f8d52ac1b90d3b15b3f0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page