LLM Open Telemetry Cost Monitor

A small local collector for token usage emitted by GitHub Copilot through OpenTelemetry. It accepts OTLP/HTTP trace protobuf and JSON, stores token usage metadata in SQLite, and estimates cost with genai-pricing.

The monitor stores trace ID, span ID, timestamp, model, input tokens, cache-read input tokens, cache-creation input tokens, and output tokens. It does not capture prompts or responses.

Requirements

• Python 3.10 or newer
• VS Code and a Copilot Chat version that supports OpenTelemetry export
• Network access for the first pricing lookup, unless genai-pricing already has a pricing cache

This project uses the repository-local virtual environment in venv/.

Install

Install the package into an activated virtual environment:

python -m pip install llm-ot-cost-monitor

To work from a repository checkout instead, install its dependencies:

./venv/bin/python -m pip install -r requirements.txt

The dependencies are pinned in requirements.txt.

Configure VS Code

Copilot's OpenTelemetry settings are application-scoped. Amend only ~/Library/Application Support/Code/User/settings.json; do not add them to a workspace .vscode/settings.json file:

{
	"github.copilot.chat.otel.enabled": true,
	"github.copilot.chat.otel.exporterType": "otlp-http",
	"github.copilot.chat.otel.protocol": "http/protobuf",
	"github.copilot.chat.otel.otlpEndpoint": "http://127.0.0.1:4318",
	"github.copilot.chat.otel.captureContent": false
}

Reload VS Code after changing the settings, then run Copilot requests while the collector is running.

Do not enable content capture unless you explicitly want prompts, responses, or source code included in telemetry. This collector does not require content capture.

Start the collector

Start the local OTLP/HTTP receiver:

llm-monitor serve --db usage.db

The llm-monitor command is installed with the package. From a repository checkout, you can instead run ./venv/bin/python monitor.py serve --db usage.db.

The receiver listens on 127.0.0.1:4318 and accepts trace exports at:

http://127.0.0.1:4318/v1/traces

To use another database, pass a different path:

llm-monitor serve --db data/usage.db

To change the listening address or port:

llm-monitor serve --host 127.0.0.1 --port 4318 --db usage.db

Keep the collector process running while using Copilot.

View usage and estimated cost

After running Copilot requests, stop or leave the collector running and execute:

llm-monitor summary --db usage.db

The output contains:

• model: reported model identifier
• requests: number of stored usage spans
• input: input tokens reported by gen_ai.usage.input_tokens
• cache_read: input tokens reported by gen_ai.usage.cache_read.input_tokens
• cache_creation: input tokens reported by gen_ai.usage.cache_creation.input_tokens
• output: total output tokens
• estimated_cost_usd: estimated cost in US dollars

Example:

model       | requests | input | cache_read | cache_creation | output | estimated_cost_usd
------------+----------+-------+------------+----------------+--------+--------------------
gpt-4o-mini |        2 |  1200 |        800 |            100 |    340 |           0.000248

Costs are calculated by genai-pricing using its pricing table, including provider-specific cache-read and cache-creation rates when available. Unknown models or unavailable prices are shown as n/a; the monitor does not guess rates.

genai-pricing treats input as the total prompt token count and subtracts the two cached-token counts before applying the regular input rate. This matches telemetry payloads where gen_ai.usage.input_tokens includes cached tokens. If an extension emits only non-cached tokens in that attribute, its estimate may undercount regular input because the semantic convention permits either payload interpretation.

Data and duplicate handling

The default database is usage.db in the repository root. SQLite creates it automatically when the collector starts.

Each record is keyed by trace ID and span ID. Repeated exports of the same span are ignored, preventing duplicate cost totals.

The database may contain sensitive operational metadata such as model names and usage volume. Keep it local and do not commit usage.db to source control.

Test

Run the unit tests with the local virtual environment:

env -u PYTHONHOME -u PYTHONPATH ./venv/bin/python -m unittest -v

The tests cover OTLP usage extraction, duplicate span handling, and the genai-pricing API call.

Limitations

• Exact usage depends on Copilot emitting the relevant gen_ai.usage.* token attributes.
• Missing token attributes cannot be reconstructed from trace metadata alone.
• Costs are estimates based on the pricing data available to genai-pricing and may not match account-specific billing, discounts, or included usage.
• The receiver is intentionally local and minimal; it does not provide authentication, a web dashboard, or remote storage.