tokenstretcher 1.1.0

TokenStretcher — Hierarchical AI task manager that breaks complex prompts into specialized agents for massive token & cost savings

TokenStretcher

A hierarchical AI task orchestrator that delivers higher quality results for a fraction of the token cost.

Instead of throwing your entire complex prompt at the most expensive model, TokenStretcher intelligently decomposes it, routes each piece to the cheapest competent specialist, executes in parallel where safe, and synthesizes a superior final answer.

Typical real-world savings: 40–70% on complex multi-part tasks.

---

Why TokenStretcher Exists

Large prompts to powerful models are brutally expensive and often produce bloated or unfocused output.

TokenStretcher wins by intelligently breaking down complex tasks and routing work to the most suitable models for the job.

---

Prepay Model — Zero Risk for You

You preload money. You can never get a surprise bill.

This is the core safety promise of TokenStretcher in production use.

How It Works

1. You add funds to your prepaid wallet (tokenstretcher topup or via Stripe/Lemon Squeezy).
2. When you run a task, TokenStretcher checks your balance first.
3. Every LLM call (via the optional LiteLLM proxy) is tracked and deducted in real time.
4. If your balance is too low, it gracefully refuses to run and tells you exactly how much to add.

Local vs Proxy Mode

• Local / BYOK mode (default): You use your own API keys. No prepay required. Great for individuals and testing.
• Proxy Mode (recommended for teams/production): All traffic goes through a LiteLLM proxy. Virtual keys with hard max_budget are issued. The proxy automatically deducts from your prepaid wallet.

You can switch between the two at any time.

Wallet Commands

tokenstretcher balance                 # See current prepaid balance + history
tokenstretcher topup                   # Show payment instructions
tokenstretcher add-funds 25            # Manually credit (local or after receiving payment)
tokenstretcher proxy start             # Start the budget-enforcing proxy server

This model completely eliminates the risk of runaway costs that has hurt many developers using raw LLM APIs.

---

Setting Up Your Proxy + Virtual Keys (Monetization Mode)

This is how you turn TokenStretcher into a real business that sells prepaid AI access safely.

Step-by-step Setup

1. Get your master key

   export XAI_API_KEY="xai-your-real-key-here"
   

2. Configure the proxy Edit .tokensaver/config.toml:

   [models]
   powerful = "xai/grok-4"
   
   proxy_default_model = "xai/grok-3"
   email_provider = "smtp"
   email_from = "[e-mail]"
   

3. Start the proxy server

   tokenstretcher proxy start
   # Then in another terminal:
   uvicorn tokensaver.proxy.server:app --port 8000
   

4. Create and deliver virtual keys

   tokenstretcher proxy create-key [e-mail] 50
   

   This will:

   • Generate a virtual key (tsai_...)
   • Link it to the user's prepaid balance
   • Automatically email the key to the customer (if email is configured)

5. Users use the virtual key They point their OpenAI client at your proxy:

   from openai import OpenAI
   
   client = OpenAI(
       base_url="http://your-proxy:8000/v1",
       api_key="tsai_xxxxxxxxxxxxxxxxxxxx"
   )
   

Every call is authenticated, budget-checked, and deducted from their prepaid balance in real time.

How Billing Works

• You control the real XAI_API_KEY
• Customers only ever receive virtual keys with limited budgets
• When their prepaid balance hits zero, their key stops working automatically
• You never have to chase invoices

This is the safest possible way to sell access to powerful models.

Quick Commands (After Setup)

# Start the full server (now launches automatically)
tokenstretcher proxy start

# Create a key manually + email it
tokenstretcher proxy create-key [e-mail] 25

# View everything
tokenstretcher proxy dashboard

Security Recommendation: Set an ADMIN_TOKEN in your environment. All /admin/* routes will then require:

Authorization: Bearer your-admin-token

Making TokenStretcher Available on PyPI

The project is now reasonably well packaged for PyPI.

To publish:

pip install build twine
python -m build
twine upload dist/*

See DEPLOYMENT.md for more details on distribution and hosting the proxy.

---

Installation

# From source (recommended during early development)
git clone https://github.com/yourname/tokensaverai
cd TokenSaver
pip install -e .

# Or after publishing
pip install tokensaverai

API Keys

TokenStretcher uses LiteLLM, so it supports virtually every provider.

Recommended environment variables:

export XAI_API_KEY="xai-..."           # Grok models (best experience)
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GROQ_API_KEY="gsk_..."

---

Quick Start

CLI (Recommended)

# Basic usage
tokenstretcher "Build a production FastAPI service with JWT auth, user CRUD, and rate limiting"

# Use a specific powerful model for planning
tokenstretcher "Design a multi-tenant SaaS permissions system" --model grok-4

# Plan only (see the decomposition without spending money)
tokenstretcher --plan-only "Refactor the entire payment module for better testability"

# Interactive mode (great for exploration)
tokenstretcher interactive

Python API

import asyncio
from tokenstretcher import Manager, load_config

async def main():
    config = load_config()
    config.default_powerful_model = "grok-4"
    config.verbose = True

    manager = Manager(config=config)
    result = await manager.run(
        "Create a complete, secure FastAPI authentication system with refresh tokens and role-based access"
    )

    print(result.final_output)
    print("\n" + result.savings.summary())

asyncio.run(main())

---

How It Works

1. Decomposition — A frontier model (your choice of powerful model) receives an elite prompt that forces it to break the task into the smallest possible high-quality subtasks while choosing the cheapest viable model tier for each.
2. Specialized Agents — Each subtask gets a narrowly scoped expert role + only the project context that is actually relevant to it.
3. Parallel Execution — Tasks with no dependencies run concurrently (controlled by max_parallel_agents).
4. Recursive Sub-Managers — For very large scopes, a sub-manager can be spawned that performs its own decomposition.
5. Synthesis — A final balanced model combines everything into one coherent, high-quality deliverable.
6. Savings Report — You get exact numbers comparing against the cost of doing it as one giant prompt.

---

Configuration

Create .tokensaver/config.toml (or config.toml) in your project:

[models]
cheap = "gpt-4o-mini"
balanced = "xai/grok-3"
powerful = "xai/grok-4"

savings_share = 0.15
max_parallel_agents = 6
max_recursion_depth = 2
verbose = true

Full options are defined in tokensaver/models.py → TokenSaverConfig.

You can also generate an example with:

tokenstretcher --init-config

---

Example Agent Roles (Built-in)

• Python Backend Expert
• FastAPI Auth Specialist
• Code Reviewer & Optimizer
• Research Summarizer
• Technical Documentation Writer
• Frontend React/TypeScript Specialist
• SQL / Database Designer
• DevOps & Deployment Expert

Adding your own roles is trivial — see the extension guide below.

---

Savings Calculator & Monetization Foundation

Every run produces a SavingsReport:

report = result.savings
print(report.percent_saved)           # e.g. 58.3
print(report.cost_saved_usd)          # e.g. 0.074
print(report.summary())

The system already calculates:

• Baseline cost of a single powerful-model call with full context
• Actual cost of the hierarchical execution
• Percentage saved

This data is also written to savings-reports/ as JSON for later analysis or proxy billing.

Future proxy mode (not yet implemented): A hosted version can sit between you and the model providers, take a configurable cut of the savings (savings_share), and still deliver net savings to the user.

---

Grok Build / MCP Integration

TokenStretcher was built from day one to be an excellent tool for other AI systems (especially Grok Build).

Recommended Patterns

• Inside Grok Build: When a user gives you a complex, multi-part, or expensive-looking task, strongly consider calling:

  tokenstretcher "the user's exact request here"
  

• As a library inside a running agent (see Python example above).

• MCP Server (planned): A future release will ship with tokenstretcher mcp serve. Until then, wrapping Manager.run with the official MCP Python SDK takes ~30 lines.

See AGENTS.md for detailed guidance on using and extending TokenStretcher inside AI coding environments.

---

Project Structure

tokensaver/
├── __init__.py
├── cli.py                 # Rich CLI with interactive + headless modes
├── manager.py             # The orchestrator (decompose → execute → synthesize)
├── agent.py               # Narrow, context-filtered specialist agents
├── models.py              # All Pydantic models + config schema
├── prompts.py             # The secret sauce (especially DECOMPOSER)
├── config.py              # TOML/YAML + env loading
└── utils.py               # Context loading, cost math, pretty printing

examples/
templates/                 # Future: saved role templates & workflows

---

Extending TokenStretcher

Add a New Role

Edit tokensaver/prompts.py:

ROLE_TEMPLATES["Kubernetes Reliability Engineer"] = """You are a senior platform engineer..."""

Then reference it in decomposition plans.

Custom Model Routing

Override TokenSaverConfig.get_model_for_tier() or pass a custom config object.

Better Context Filtering

Improve utils.filter_context_for_task() — the current implementation is deliberately simple and cheap.

New Execution Strategies

Subclass Manager and override _execute_plan or _synthesize.

---

Development

pip install -e ".[dev]"
ruff check .
pytest

---

Roadmap

• Native MCP server (tokenstretcher mcp serve)
• Persistent task graphs + resume
• Real output token accounting + better cost tracking via LiteLLM
• Hosted proxy with savings-share billing
• Evaluation harness with golden tasks
• VS Code / Cursor extension

---

License

MIT © TokenStretcher Contributors

---

Philosophy (TL;DR)

The best AI system is not the one that uses the biggest model. It is the one that uses the right model for every piece of the work, with the least possible wasted context, and still ships outstanding results.

TokenStretcher is an early, pragmatic step in that direction.

---

Disclaimer and Limitation of Liability

IMPORTANT – READ CAREFULLY BEFORE USING THIS SOFTWARE

This software is provided "AS IS" and "AS AVAILABLE", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement.

IN NO EVENT SHALL THE AUTHORS, CONTRIBUTORS, OR COPYRIGHT HOLDERS BE LIABLE for any claim, damages, or other liability, whether in an action of contract, tort (including negligence), or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software, even if advised of the possibility of such damages.

Special Warning for Paid / Commercial Use

This software contains components (the proxy server, virtual key system, prepaid wallet, and Lemon Squeezy integration) that are specifically designed to support paid commercial services that resell or intermediate access to third-party large language models.

By operating any paid instance of this software you expressly acknowledge and agree that:

• You assume all risk and full legal responsibility for compliance with the Terms of Service of any LLM providers whose models are accessed through this software.
• The authors make no guarantees whatsoever regarding billing accuracy, key generation, email delivery, service availability, data integrity, or correctness of any financial tracking.
• You are solely responsible for compliance with all applicable laws, including consumer protection, data protection (GDPR, CCPA, etc.), and financial regulations.
• The authors shall have no liability for any financial losses, chargebacks, refunds, regulatory actions, customer disputes, service interruptions, or any other damages arising from your commercial use of this software.

Strongly Recommended: If you operate a paid service using this software, you should create and publish your own Terms of Service and Privacy Policy that govern your relationship with your customers.

See the full LICENSE file for the complete legal text.

---

License & Commercial Use

TokenStretcher is released under the Business Source License (BSL 1.1).

Summary

• Free for individuals and non-commercial / internal use.
• Commercial use is restricted until the Change Date (4 years after publication).
• You may not resell, host, or embed this as a paid service (including inside commercial AI coding agents) without a separate license.

Why This License?

The goal is to keep the tool genuinely useful and open for developers and AI agents (especially Grok Build users), while preventing the core IP — particularly the prepaid proxy model and high-quality hierarchical orchestration — from being immediately copied and resold.

If you want to build a commercial product on top of TokenStretcher (hosted proxy, enterprise agent integration, etc.), please reach out for a commercial license.

See the full LICENSE file for legal text.

---

Philosophy (TL;DR)

The best AI system is not the one that uses the biggest model. It is the one that uses the right model for every piece of the work, with the least possible wasted context, and still ships outstanding results.

TokenStretcher is an early, pragmatic step in that direction.

Contributions that improve quality or cost (ideally both) are extremely welcome.