tokenbee-sdk 1.3.0

Official Python SDK for TokenBee LLM inference gateway and observability.

TokenBee Python SDK

Official Python SDK for TokenBee - The Intelligent LLM Inference Gateway with Observability, Compression, and Privacy.

Features

• Unified API: Access multiple LLM providers (OpenAI, Anthropic, Google, Mistral, etc.) through a single interface.
• Intelligent Compression: Reduce token usage and latency with context-aware compression.
• Privacy Guard: Automatic PII masking and privacy-preserving inference.
• Built-in Observability: Automatic tracking of latency, costs, and token usage.

Links

• Homepage: https://tokenbee.io
• Dashboard: https://tokenbee.io/dashboard

Installation

pip install tokenbee-sdk

Quick Start

from tokenbee import TokenBee, TokenBeeModel, CompressionRate

# Initialize the client with your TokenBee API key 
# AND your LLM provider key (Bring Your Own Key - BYOK)
client = TokenBee(
    api_key="your_tokenbee_api_key",
    llm_key="your_llm_provider_key", # e.g. OpenAI or Anthropic key
    compression="auto",
    rate=CompressionRate.MEDIUM
)

# Send a request
response = client.send(
    model=TokenBeeModel.OPENAI_GPT_4O,
    input={
        "messages": [
            {"role": "user", "content": "Explain quantum entanglement in simple terms."}
        ]
    }
)

print(response["choices"][0]["message"]["content"])

Bring Your Own Key (BYOK)

TokenBee is a stateless gateway. We do not store your LLM provider API keys in our database. You pass your provider key (OpenAI, Anthropic, etc.) through the SDK's llm_key parameter. The SDK sends this in the X-LLM-Key header, allowing the TokenBee proxy to forward requests to the provider on your behalf while you maintain full control over your billing and security.

Advanced Usage

Compression Control

You can specify the compression rate and method per request. TokenBee uses an intelligent semantic engine to reduce token usage while preserving meaning.

response = client.send(
    model=TokenBeeModel.ANTHROPIC_CLAUDE_3_5_SONNET,
    input={
        "messages": [...],
        "compression": "auto",      # "auto" (default), "on", or "off"
        "rate": CompressionRate.HIGH, # MEDIUM (0.5), HIGH (0.33), etc.
        "privacy": True
    }
)

• compression: Set to "auto" to let TokenBee decide when to compress, or "off" to bypass the compression engine entirely for high-precision tasks.
• rate: Controls the aggressiveness of compression. HIGH aims for ~67% token reduction.
• sessionId: (Optional) String ID to group multiple requests into a single replayable session in the dashboard.
• userId: (Optional) String ID to track usage and costs per unique end-user.
• privacy: (Optional) Set to True to disable payload logging and session replays for this request. Metadata (latency, tokens) will still be recorded for observability.

Supported Models

The SDK provides a TokenBeeModel enum with popular models:

• TokenBeeModel.OPENAI_GPT_4O
• TokenBeeModel.ANTHROPIC_CLAUDE_3_5_SONNET
• TokenBeeModel.GEMINI_2_0_FLASH
• ... and many others.

License

MIT