⚡ TokenShrink

A lightweight Python library for prompt compression and token optimization—reduce LLM input costs by 30-50% without sacrificing context.

Features

• Prompt Pruning: Automatically strips conversational fluff ("please", "kindly") and collapses unnecessary whitespace.
• Data Serialization: Converts heavy JSON payloads into token-efficient tabular representations.
• Zero Overhead: Blazing fast execution using pure Python regular expressions.

Quickstart

pip install -e .

from token_shrink import prune_prompt, compress_struct

# Prune filler words
raw_prompt = "Could you please kindly analyze this data?"
clean_prompt = prune_prompt(raw_prompt)
print(clean_prompt)  # "analyze this data"

# Compress structured data
data = [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
compact = compress_struct(data)
print(compact)
# [2]{id,name}:
# 1,Alice
# 2,Bob

pytest