truthprobe 0.1.0

AI API Trust & Cost Transparency SDK — Audit your LLM relay providers

TruthProbe SDK

AI API Trust & Cost Transparency — Know if your relay provider is honest.

Install

pip install truthprobe

Quick Start

import truthprobe
truthprobe.patch()

# Use OpenAI SDK as normal — TruthProbe monitors in background
from openai import OpenAI
client = OpenAI(api_key="sk-xxx", base_url="https://your-relay.com/v1")
response = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Hello"}]
)
# After each response, you'll see:
# ✓ ¥0.032 │ claude-sonnet-4-6 │ 1.2s

CLI

truthprobe report      # Full audit report
truthprobe balance     # Check provider balances
truthprobe score       # Current trust score

Configuration

import truthprobe

truthprobe.init(
    providers=[
        {"name": "RelayA", "base_url": "https://relay-a.com/v1", "key": "sk-aaa"},
        {"name": "RelayB", "base_url": "https://relay-b.com/v1", "key": "sk-bbb"},
    ],
    lang="zh",                    # "zh" or "en" (auto-detected by default)
    alert_balance_threshold=50,   # Alert when balance < ¥50
    currency_symbol="¥",
)

truthprobe.patch()

Silent Mode

truthprobe.patch(quiet=True)    # No output at all, just records
truthprobe.patch(verbose=False) # No per-request line, but alerts still fire

What It Detects

• Model downgrade: Relay claims to use GPT-4o but serves GPT-4o-mini
• Timing anomalies: Response too fast for the claimed model
• Text quality drops: Vocabulary richness / reasoning depth below model baseline

Alerts (automatic, no config needed)

1. Balance danger: When any provider balance < threshold
2. Trust collapse: When suspicious rate > 30% in recent 20 requests

License

MIT