Skip to main content

AI agent cost observability โ€” track spending, latency, and token usage across all your agent sessions

Project description

๐Ÿ“Š Bonanza Observe

AI agent cost observability โ€” track spending, latency, and token usage across all your agent sessions

PyPI Python License

Stop guessing how much your AI agents cost. Bonanza Observe gives you real-time cost tracking, token usage analytics, and latency monitoring for every LLM call.

Why?

AI agents make dozens of LLM calls per task. Without observability:

  • You don't know what you're spending until the bill arrives
  • You can't optimize what you can't measure
  • You can't budget without tracking

Bonanza Observe is the missing open() for AI agent economics.

Installation

pip install bonanza-observe

Quick Start

from bonanza_observe import Tracker

# Create a tracker with a budget
tracker = Tracker(name="my-agent", budget_usd=10.00)

# Start tracking
session = tracker.start_session("research-task")

# Track LLM calls
span = tracker.start_span("web-search")
tracker.record_cost(
    provider="openai",
    model="gpt-4o",
    input_tokens=1500,
    output_tokens=800,
    latency_ms=1200,
)
tracker.finish_span()

# Check your budget
print(tracker.check_budget())
# {
#   "total_usd": 0.01175,
#   "budget_usd": 10.00,
#   "remaining_usd": 9.98825,
#   "percentage": 0.12,
#   "over_budget": False,
#   "alert": False
# }

# Get a full summary
print(tracker.summary())
# {
#   "total_cost_usd": 0.01175,
#   "total_tokens": 2300,
#   "by_model": {"gpt-4o": {"cost": 0.01175, "calls": 1}},
#   "by_provider": {"openai": {"cost": 0.01175, "calls": 1}},
#   ...
# }

Features

๐Ÿ’ฐ Cost Tracking

  • Per-model, per-provider cost breakdown
  • Auto-calculates cost from token counts (supports GPT-4o, Claude 3.5, Gemini, Llama, and more)
  • Manual cost override for custom pricing
  • Budget alerts at configurable thresholds (default 80%)

๐Ÿ“Š Token Usage

  • Input/output/total token tracking per call
  • Aggregate stats across sessions
  • Per-model token breakdown

โฑ๏ธ Latency Monitoring

  • Per-call latency tracking
  • Average latency across sessions
  • Nested span timing

๐Ÿ“ Session Management

  • Group calls into sessions and spans
  • Nested spans for complex agent workflows
  • Tags for filtering and grouping

๐Ÿ“ค Export

from bonanza_observe import Tracker, JsonExporter, CsvExporter

tracker = Tracker(name="my-agent")
# ... track calls ...

# Export as JSON
json_str = JsonExporter.export(tracker)
JsonExporter.save(tracker, "costs.json")

# Export as CSV
csv_str = CsvExporter.export(tracker)
CsvExporter.save(tracker, "costs.csv")

Supported Models (Pricing)

Model Input (per 1M tokens) Output (per 1M tokens)
GPT-4o $2.50 $10.00
GPT-4o Mini $0.15 $0.60
GPT-4 Turbo $10.00 $30.00
Claude 3.5 Sonnet $3.00 $15.00
Claude 3 Haiku $0.25 $1.25
Claude 3 Opus $15.00 $75.00
Gemini 1.5 Pro $1.25 $5.00
Gemini 1.5 Flash $0.075 $0.30
Llama 3.1 70B $0.60 $0.60
Llama 3.1 8B $0.05 $0.05

Comparison

Feature Bonanza Observe Revenium Helicone LangSmith
Cost tracking โœ… โœ… โœ… โœ…
Token tracking โœ… โœ… โœ… โœ…
Latency monitoring โœ… โœ… โœ… โœ…
Budget alerts โœ… โœ… โŒ โœ…
Session/span grouping โœ… โŒ โŒ โœ…
JSON/CSV export โœ… โœ… โœ… โœ…
Nested spans โœ… โŒ โŒ โŒ
Auto cost calculation โœ… โœ… โœ… โŒ
Zero dependencies โœ… โŒ โŒ โŒ
Self-hosted โœ… โœ… โŒ โŒ
Python-native โœ… โŒ โœ… โœ…

Requirements

  • Python 3.10+

License

Apache License 2.0 โ€” see LICENSE for details.

Links


Built by Bonanza Labs ๐Ÿ“Š

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bonanza_observe-0.1.0.tar.gz (9.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

bonanza_observe-0.1.0-py3-none-any.whl (7.4 kB view details)

Uploaded Python 3

File details

Details for the file bonanza_observe-0.1.0.tar.gz.

File metadata

  • Download URL: bonanza_observe-0.1.0.tar.gz
  • Upload date:
  • Size: 9.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.5

File hashes

Hashes for bonanza_observe-0.1.0.tar.gz
Algorithm Hash digest
SHA256 8d9fa22b5e7d51c9da93b22ae779d37e6ea84671ac456a068a3e679d3f50c9b1
MD5 ee653fe33b710c2cd2a790ce9a6e5902
BLAKE2b-256 e791e6ee4640980b313663695615bbd9ea84d0c2f9695594bb9237832feeb34b

See more details on using hashes here.

File details

Details for the file bonanza_observe-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for bonanza_observe-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 a219e31189dc5cc88a6adc9bc218302183da3612de1670b18782a81780c51b30
MD5 734e1e3aea95faaf185d6c7c1b975958
BLAKE2b-256 bb5cc95184e447ba2a241bad3eb14df18db840074dbdf074fdfb73a0c525815b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page