Token Optimizer Plugin
Version: 1.0.0
Author: Aveek Sengupta (aveekforyou@gmail.com)
License: public
A local-first, prompt-only, end-to-end token-optimization toolkit for VS Code Copilot. It attacks both halves of a turn's token cost:
- Input side — compress large workspace context fed INTO the agent into a reversible, traceable context pack (
temp/context_pack_vN/), so the agent reasons over compressed context with full source traceability. - Output side — rewrite the agent's own replies coming OUT into lean, high-signal language ("Shrink the mouth, not the brain").
One inviolable invariant spans both sides: code, commands, file paths / URLs / identifiers, error & log lines, and numbers / versions stay byte-for-byte exact. Everything runs as prompt instructions + deterministic stdlib Python helpers — no network calls, no telemetry, no external LLM APIs. The agent is the LLM.
Components
Three Skills + Three Agents
| Component | Kind | Side | Role |
|---|---|---|---|
token-optimizer |
skill | input+output | Combined entrypoint; activates both sides; delegates all real work to the two sibling skills, adds no new compression logic. Supports one-shot and session mode. |
input-token-optimizer |
skill | input | Builds the context pack (5 artifacts) under temp/context_pack_vN/. |
output-token-optimizer |
skill | output | Lean output-style rewriter + companion actions (speak, lean-commit, lean-review, lean-compress-file). |
token-optimizer-agent |
agent | input+output | Thin wrapper → token-optimizer skill → combined YAML verdict. |
input-token-optimizer-agent |
agent | input | Thin wrapper → input-token-optimizer skill → input YAML verdict. |
output-token-optimizer-agent |
agent | output | Thin wrapper → output-token-optimizer skill → output YAML verdict. |
Input-Side Compression Modes & Savings Bands
| Mode | Behaviour | Savings Band |
|---|---|---|
quick |
Aggressive compression, precision over coverage | 60–80% |
balanced (default) |
Blend of coverage and compression | 40–60% |
lossless |
Broader detail retention, lower savings | 20–40% |
Required Input Artifacts (temp/context_pack_vN/)
context_pack.md— task-focused compressed context (human-readable).context_manifest.json— reversible chunk → source mapping (100%manifest coverage).context_retrieval_index.json— tag/symbol/file/time lookup.context_stats.md— compression statistics + confidence metrics.context_pack_reasoning_log.md— audit trail of compression decisions.
Output-Side Compression Levels
| Level | Behaviour | Example |
|---|---|---|
lite |
Full sentences; no preamble/hedging/question-restatement | "Wrap the object in useMemo; a new reference is created every render." |
full (default) |
Sentence fragments, imperative voice, minimal connectives | "New ref each render. Wrap object in useMemo." |
ultra |
Telegraphic fragments, symbols over words | "New ref/render — useMemo it." |
Say normal mode, verbose, or stop terse at any time to revert.
Installation (PyPI)
pip install token-optimizer-plugin
Python API Usage
from token_optimizer import compute_stats, savings_report
# Output-side savings & invariant check
report = savings_report(
"Great question! Please run `pytest -q` in src/app/main.ts on v1.2.0.",
"Run `pytest -q` in src/app/main.ts (v1.2.0).",
)
print(report)
# Input-side context pack stats
stats = compute_stats(raw_chars=400, compressed_chars=100)
print(stats.savings_percent) # 75.0
CLI Usage
# Output-side savings report
token-optimizer savings --raw "Please run `npm test` on v1.0.0" --lean "Run `npm test` (v1.0.0)"
# Input-side context pack stats
token-optimizer stats --raw-chars 400 --compressed-chars 100
# Rank files for a goal
token-optimizer rank --goal "error triage" README.md
# Type-aware file compression
token-optimizer compress --goal "optimize" skills/input-token-optimizer/scripts/context_pack_builder.py
Runtime Requirements & Testing
- Python 3.9+ (stdlib only — no third-party runtime dependencies).
- Run the full aggregate test suite (33 tests across both sibling suites) from the plugin root:
python skills/token-optimizer/assets/tests/run_all_tests.py
Metadata
Release files for token-optimizer-plugin 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| token_optimizer_plugin-1.0.0.tar.gz | 34.7 kB | With provenance | Details for token_optimizer_plugin-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| token_optimizer_plugin-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for token_optimizer_plugin-1.0.0-py3-none-any.whl |
Total release size: 45.8 kB
Release files / token_optimizer_plugin-1.0.0.tar.gz
| Download URL | token_optimizer_plugin-1.0.0.tar.gz |
|---|---|
| Size | 34.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d5cb7e0e6644dfb14b73000176d0de8ee677ac024d6abcbe4b8a3473b1214479
|
|
BLAKE2b-256 checksum How to use checksums |
21fa765efc0d56bf6b8fb2230e194992c12463afe24c1f5c8013975d10f87ce2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.
Transparency logRelease files / token_optimizer_plugin-1.0.0-py3-none-any.whl
| Download URL | token_optimizer_plugin-1.0.0-py3-none-any.whl |
|---|---|
| Size | 11.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
723eb298a284839c1a1da940415983c46a864c5db11af58275eb2257d667ee50
|
|
BLAKE2b-256 checksum How to use checksums |
d3273871a186fa93d55c67cf86956f22ce6dcd85672b57b1fb4cba15d6afc12e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.
Transparency logSecurity for token-optimizer-plugin 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for token_optimizer_plugin-1.0.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 17, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.