Skip to main content

A semantic token optimizer for LLM prompts — reduce tokens, preserve meaning

Project description

🧵 TokenMesh

Why send 8000 tokens when 3000 do the same job?

GitHub · Live Demo


What is this?

TokenMesh is a Python library that optimizes LLM prompts before sending them to APIs.

It removes:

  • duplicate instructions
  • repeated content
  • filler text

while preserving meaning, constraints, and rules.


Why use it?

LLM prompts are often verbose and redundant, which:

  • increases token cost
  • slows responses
  • hits context limits

TokenMesh helps you:

  • reduce 40–75% tokens
  • lower API costs
  • keep prompts clean and efficient

Same meaning → fewer tokens → better performance.


Install

pip install tokenmesh
  • Python 3.9+
  • Works on Windows, Mac, Linux

Quick Start

from tokenmesh import TokenMesh

tm = TokenMesh()

result = tm.optimize(
    text=your_prompt,
    query="optional"
)

print(result.optimized_text)
print(result.reduction_percent)

Claude Integration

from tokenmesh.integrations.claude import TokenMeshClaude

client = TokenMeshClaude()

response = client.chat(
    system=long_prompt,
    user="your question",
    model="claude-sonnet-4"
)

print(response.content)

Modes

Lite (safe)

  • 20–40% reduction
  • preserves strict rules
from tokenmesh import TokenMeshLite
result = TokenMeshLite().optimize(text)

Aggressive (max compression)

  • 55–75% reduction
  • best for long content
from tokenmesh import TokenMeshAggressive
result = TokenMeshAggressive().optimize(text, query="...")

Example

Before: ~684 tokens After: ~290 tokens

Same instructions. ~57% fewer tokens.


License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenmesh-0.2.1.tar.gz (20.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tokenmesh-0.2.1-py3-none-any.whl (19.9 kB view details)

Uploaded Python 3

File details

Details for the file tokenmesh-0.2.1.tar.gz.

File metadata

  • Download URL: tokenmesh-0.2.1.tar.gz
  • Upload date:
  • Size: 20.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for tokenmesh-0.2.1.tar.gz
Algorithm Hash digest
SHA256 897095bf2969126fa8c464fbd125bd5e8da89691217a8ba6158b8d8f272c9bab
MD5 112ddcdcdd57a495da99ad2a1180d402
BLAKE2b-256 977d3ef76e60fcee799c0106331aa1362283f9ecd47f71c34a560a5927c77113

See more details on using hashes here.

File details

Details for the file tokenmesh-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: tokenmesh-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 19.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.7

File hashes

Hashes for tokenmesh-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 af2c82139b9faf08ec8a0dcc931d4adbecbe314b4918beea6e3e033c0fe38c91
MD5 9128994c675c5b1bc78c69b951c5c406
BLAKE2b-256 4da4715421f0c64f64a4e1c38af127538cb5b77ff2313ab914fa8fe1ec6ccbb8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page