fewertokens

Python SDK for FewerTokens — a control-plane client for cross-family LLM routing.

The SDK asks FewerTokens which provider/model to use, then calls that model locally via LiteLLM with your provider keys. Usage is reported back asynchronously.

Install

pip install fewertokens

Quick start

from fewertokens import FewerTokens

ft = FewerTokens(
    api_key="tr_live_...",                         # your FewerTokens key from the dashboard
    openai_api_key="sk-proj-...",
    openai_default_model="gpt-4o",                 # your current default — savings are measured against this
    anthropic_api_key="sk-ant-...",
    anthropic_default_model="claude-sonnet-4-6",   # priced default so routing can save money
    gemini_api_key="AIzaSy...",
    gemini_default_model="gemini-3.1-pro-preview", # priced default so routing can save money
    base_url="https://api.fewertokens.in",
)

response = ft.chat.completions.create(
    messages=[{"role": "user", "content": "What is Python?"}],
)

# The answer
print(response.content)

# Which model was actually used (may be cheaper than your default)
print(f"Routed to: {response.routing.provider}/{response.routing.model_used}")
print(f"Baseline was: {response.routing.baseline_model}")

# Token usage
print(f"Tokens — prompt: {response.usage.prompt_tokens}, completion: {response.usage.completion_tokens}")

# Estimated savings for this request (USD)
print(f"Estimated saving this call: ${response.routing.estimated_savings_usd:.6f}")

Response object

ft.chat.completions.create(...) returns a ChatResponse — not the OpenAI SDK object. Key attributes:

How it works

1. Route handshake — POST /v1/route with the routing prompt (your system messages + the last user message only; the rest of the conversation is never sent, see "What FewerTokens receives" below) + provider_models (each provider key's default model). The whole handshake (DNS, connect, send, receive) has one 800 ms deadline.
2. Local dispatch — litellm.completion / acompletion with the routed provider key
3. Telemetry — background POST /v1/telemetry with route_id + token usage. The completion call does not wait, and a failed report does not raise. close(), the context manager (with and async with), and process exit wait up to 2 seconds for a report still in flight. That wait does not raise.

If the control plane is unreachable, slow, or rejects the request (including an invalid FewerTokens API key), the SDK calls your first configured *_default_model locally and skips telemetry, so there is no fee. FewerTokens must not be the reason a prompt never reaches the LLM provider. response.routing.fallback_reason is "route_unavailable" in that case.

When an invoice hasn't been paid, the response comes back as HTTP 402 with error code routing_paused. The SDK logs one warning per client on the fewertokens.chat logger: "FewerTokens routing is paused because an invoice hasn't been paid. Pay it using the link in your invoice email to turn routing back on." That call goes to your default model and sends no usage telemetry, so no fee is charged.

If the routed model fails on the provider side (5xx, overloaded, 429 rate limit, timeout, connection error), the SDK retries once on your first configured *_default_model. Telemetry marks the call as a fallback, so no fee is charged, and response.routing.fallback_reason is "routed_model_error". Errors caused by the request or your account (400/422, 401/403, 404, content policy, context length) are raised without a retry. For streams, the retry only happens if the routed stream fails before its first chunk; once tokens have reached your code, a mid-stream error is raised.

Privacy & Security

🔒 Privacy & Security First:

• Zero Key Exposure: Your provider keys stay strictly in your client runtime's RAM memory and are never sent to FewerTokens servers.
• Direct Local Execution: Prompt content and response streams flow directly between your machine and LLM provider endpoints (OpenAI, Anthropic, Gemini, etc.). FewerTokens only receives the routing prompt (your system messages + the last user message, see "What FewerTokens receives" below) for the routing decision (~20ms) and never sees or logs completion payloads.

Streaming

for chunk in ft.chat.completions.create(
    messages=[{"role": "user", "content": "Explain Docker"}],
    stream=True,
):
    print(chunk.content, end="", flush=True)

Async

response = await ft.chat.completions.acreate(
    messages=[{"role": "user", "content": "Hello!"}],
)

Notes

• At least one of openai_api_key, anthropic_api_key, or gemini_api_key is required, and each key must have a matching *_default_model.
• String shorthand: prompt="Hello" is accepted as an alternative to messages=.
• model= on .create() is ignored — the control plane decides the model.
• Your *_default_model is the savings baseline: set it to the model you are currently calling. If you set it to an already-cheap model, there is nothing to save and the fee is $0.
• FewerTokens never sees your provider API keys or completion payloads. Keys stay in your process RAM.

What FewerTokens receives

Since 0.3.0, POST /v1/route carries exactly three fields: prompt, available_providers and provider_models.

• prompt is built locally from your messages:
  • all of your system messages, joined;
  • plus the last user message only;
  • only text parts are used; images and other non-text parts are dropped;
  • if there is no user message at all, the last message of any role is used instead.
• Earlier user turns, assistant replies and tool results are never sent. The full conversation goes only to your LLM provider, through LiteLLM on your machine. (0.2.x sent the whole messages list as well.)
• The server's classifier reads at most the first 256 tokens of prompt.
• The SDK sends at most the first 8,192 characters of prompt. A prompt longer than 32,000 characters is rejected by the API. The SDK then calls your default model directly: there is no routing and no fee.
• available_providers and provider_models carry your provider names and each provider's *_default_model id.
• POST /v1/telemetry carries the route_id and token counts. For a retried call it also carries fallback, fallback_provider and fallback_model. It never carries prompt or completion text.

Default model has no verified price

You may see this warning once per process, on the fewertokens.pricing logger:

Your default model has no verified price, so routing and fees are off. Pick a priced default to turn them on. See https://pypi.org/project/fewertokens/#user-content-default-model-has-no-verified-price

What it means. Your default model is the *_default_model of the first provider you configure (OpenAI, then Anthropic, then Gemini, then extra_providers in order). FewerTokens has no verified price for it, so it can't measure savings against it. Your calls still work: every call goes straight to your default model, with no routing and no fee. The SDK never switches your default model for you.

The SDK logs the warning when you create the client and your default model isn't in its price table, or when POST /v1/route reports routing_disabled_reason: "default_model_unpriced".

How to fix it. Your default model is set in your SDK config, not on the dashboard. Pass a priced model as the *_default_model of your first provider:

from fewertokens import FewerTokens

ft = FewerTokens(
    api_key="tr_live_...",
    anthropic_api_key="sk-ant-...",
    anthropic_default_model="claude-sonnet-4-6",  # was "claude-3-5-sonnet", which has no verified price
)

For providers in extra_providers, set "default_model" in that provider's entry.

How to check whether a model is priced. The SDK's price table holds the models in the published FewerTokens catalog (1.2.1). Priced defaults include gpt-4o, gpt-4.1 and gpt-5.5 (OpenAI), claude-sonnet-4-6 and claude-opus-4-7 (Anthropic), and gemini-3.1-pro-preview (Gemini). Models the catalog dropped, such as claude-3-5-sonnet, o1-mini, gpt-5.6, gemini-2.0-flash, grok-4, and (in 1.2.1) gemini-2.5-pro, gemini-2.5-flash, o1 and o4-mini, have no verified price, and neither do fine-tunes or other models outside the catalog.

To check your own config, create the client with logging on. If the warning doesn't appear when you create the client or on your first call, your default model is priced:

import logging

logging.basicConfig(level=logging.WARNING)
ft = FewerTokens(api_key="tr_live_...", openai_api_key="sk-proj-...", openai_default_model="gpt-4o")

How to silence it or route it. The SDK adds no log handlers. The warning goes through Python's standard logging, so your logging config decides where it ends up. Without any config, Python prints it to stderr.

import logging

# Silence it:
logging.getLogger("fewertokens.pricing").setLevel(logging.ERROR)

# Or send it to your own handler instead of the root logger:
pricing_log = logging.getLogger("fewertokens.pricing")
pricing_log.addHandler(logging.FileHandler("fewertokens.log"))
pricing_log.propagate = False

Silencing the warning doesn't turn routing back on. Only a priced default model does that.