forge-dashboard 0.1.0

Observability dashboard for LLM inference with thinking budget and mode tracking

forge-dashboard

Observability dashboard for LLM inference with reasoning mode support. Ingests OpenTelemetry trace and metric data from forge-observe, stores it in SQLite, and serves a React frontend for visualizing thinking budget burn, token splits, and mode switches.

What it shows

• Budget burn over time -- how fast the thinking budget is being consumed
• Think vs response token split -- what fraction of output is thinking vs user-facing
• Mode switch timeline -- when the router flips between thinking and instruct mode
• Alert rules -- threshold alerts on budget exhaustion, token ratios, error rates

Architecture

forge-observe (OTel SDK)
        |
        | OTLP/HTTP JSON (POST /v1/traces, /v1/metrics)
        v
forge-dashboard (FastAPI + SQLite)
        |
        +-- React frontend (Vite + Recharts + Tailwind)

The backend accepts OTLP JSON exports, parses forge-specific span attributes (forge.thinking_tokens, forge.budget_remaining, etc.), and stores them for dashboard queries. Authentication is via API keys scoped to projects.

There are two ways to send data:

• Direct -- point forge-observe's OTLP exporter straight at http://<dashboard-host>:8000/v1/traces. Works for single-node setups.
• Via forge-cloud -- if you run forge-cloud as a central proxy, configure it to forward traces to the dashboard. forge-cloud handles auth and routing so individual services only need the proxy address.

Running locally

# backend
pip install -e ".[dev]"
forge-dashboard
# serves on http://localhost:8000

# frontend (separate terminal)
cd frontend
npm install
npm run dev
# serves on http://localhost:5173, proxies /v1/* to backend

Setup

1. Start the backend. It creates a SQLite database on first run.
2. Set a setup secret and create a user and project via the API:

export FORGE_SETUP_SECRET="your-secret-here"
# then start the backend

# create user
curl -X POST http://localhost:8000/v1/users \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-secret-here" \
  -d '{"email": "[e-mail]"}'

# create project (use the user id from above)
curl -X POST http://localhost:8000/v1/projects \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-secret-here" \
  -d '{"name": "my-project", "owner_id": 1}'
# response includes api_key -- save it

3. Enter the API key in the dashboard Settings page, or use it in forge-observe's exporter config.
4. Point forge-observe's OTLP exporter at http://localhost:8000/v1/traces.

Configuration

Environment variables:

Variable	Default	Description
SQLITE_PATH	forge_dashboard.db	Path to SQLite database file
FORGE_SETUP_SECRET	(none)	Required to create users/projects via API
FORGE_ALERT_INTERVAL_SECONDS	60	How often to evaluate alert rules
FORGE_RETENTION_DAYS	7	Data retention period
FORGE_CORS_ORIGINS	http://localhost:5173	Comma-separated allowed CORS origins

Tests

pip install -e ".[dev]"
pytest -v

Project status

Backend is functional with OTLP ingestion, query API, alerting, and data retention. Frontend is a working prototype with budget, token split, mode timeline, and alert management views. Notification delivery (email, Slack, etc.) is not yet implemented -- alert firings are recorded in the database only.