datasette-otel-otlp-exporter

Send Datasette's builtin OpenTelemetry traces to any OTLP backend: Jaeger, Grafana Cloud, Honeycomb, an OpenTelemetry Collector, and so on.

Datasette records a span for every request and SQL query, but it doesn't send them anywhere on its own. datasette-otel-otlp-exporter sends them over OTLP/HTTP, the protocol nearly every tracing backend accepts, so you can see where a slow page spends its time, down to individual SQL queries. You configure it with a single plugin setting; there's no opentelemetry-instrument wrapper to run.

Installing

This plugin requires Datasette 1.0a41 or higher.

uv add datasette-otel-otlp-exporter

Usage

A one-liner with uvx, sending traces to an OTLP endpoint on localhost:4318:

uvx --prerelease=allow \
  --with datasette-otel-otlp-exporter \
  --with 'datasette>=1a41' \
  datasette \
    -s plugins.datasette-otel-otlp-exporter.endpoint http://localhost:4318 \
    my_data.db

To try this locally, run Jaeger, which ships as a single binary and accepts OTLP on port 4318. Load a few Datasette pages, wait about five seconds for the next batch to go out, then open the Jaeger UI at http://localhost:16686 and pick the datasette service:

Hosted backends usually want an API key in a header. Keep the key out of your config file with Datasette's $env substitution:

# datasette.yaml
plugins:
  datasette-otel-otlp-exporter:
    endpoint: https://api.honeycomb.io
    headers:
      x-honeycomb-team:
        $env: HONEYCOMB_API_KEY

HONEYCOMB_API_KEY=... datasette my_data.db -c datasette.yaml

Grafana Cloud

Grafana Cloud's free tier is a cheap way to get a hosted trace UI, for example for a Datasette instance on Fly.io. Its OTLP gateway needs a specific URL and a basic-auth header, so the plugin builds both from a grafana_cloud block:

# datasette.yaml
plugins:
  datasette-otel-otlp-exporter:
    grafana_cloud:
      region: prod-us-east-0
      instance_id: "123456"
      api_token:
        $env: GRAFANA_CLOUD_TOKEN

All three values come from the OpenTelemetry tile on your stack's page at grafana.com, and the token needs the traces:write scope (creating one). Newer Grafana Cloud regions use a different gateway hostname than otlp-gateway-<region>.grafana.net. If yours does, replace region with endpoint set to the full URL from the tile, ending in /otlp/v1/traces.

Service name and sampling

These use the standard OpenTelemetry environment variables:

OTEL_SERVICE_NAME=my-datasette \
OTEL_TRACES_SAMPLER=parentbased_traceidratio \
OTEL_TRACES_SAMPLER_ARG=0.25 \
  datasette my_data.db -c datasette.yaml

This labels the instance my-datasette in your tracing UI and keeps a quarter of traces. By default every trace is kept and the service name is datasette.

More details are in the Reference below.

Privacy

Spans include the text of every SQL query Datasette runs, in the db.query.text attribute. On a public instance that includes any SQL visitors type into the query editor or send with ?sql=. Parameter values are never recorded, but table names, column names and literal values written into the SQL are. Everything goes to the endpoint you configure, so if that's a third-party service, you are sending it your users' queries. Treat your tracing backend with the same care as the database itself.

Reference

Options

# datasette.yaml
plugins:
  datasette-otel-otlp-exporter:
    endpoint: http://localhost:4318
    headers:
      x-api-key:
        $env: TRACING_API_KEY

grafana_cloud takes:

Some behavior to be aware of:

• Without an endpoint (or grafana_cloud block), the plugin prints no endpoint configured - OpenTelemetry export is disabled to stderr at startup and sends nothing.
• Unknown options and invalid values fail at startup, listing every problem:

  Error: datasette-otel-otlp-exporter: invalid plugin config
    endpiont: Extra inputs are not permitted
  

• An explicit endpoint takes precedence over the one grafana_cloud builds. Explicit headers are merged over its Authorization header, key by key.
• Spans are sent in batches, about every five seconds, and once more when Datasette exits.
• If the backend is unreachable, Datasette keeps serving pages as normal. Each batch is retried for a few seconds, then dropped with a Failed to export span batch log message.

Environment variables

The standard OpenTelemetry environment variables are supported, and they take precedence over plugin config:

The service name and sampler are read when Datasette loads its plugins, so they also apply to the datasette.startup trace.

What gets exported

Whatever Datasette records. The plugin adds no spans of its own. That's one span per HTTP request named after the method and the route that matched it, one per SQL query with its text, database and timing, spans for writes, and a datasette.startup trace covering plugin hooks and loading the internal catalog. The full list of spans and attributes is in Datasette's telemetry documentation.

Only traces are exported. For metrics, see datasette-otel-prometheus.

Using alongside other OpenTelemetry setups

OpenTelemetry allows one tracer provider per process. The plugin installs its own when it is first imported, unless one already exists. If Datasette runs under opentelemetry-instrument, inside an application that set up tracing, or alongside another exporter plugin such as datasette-otel-file-exporter, the plugin adds its exporter to that provider instead. Both get every span, whichever loaded first. In that case the other provider's sampler and service name apply.

If the existing provider isn't the OpenTelemetry SDK's (for example a NoOpTracerProvider, which turns tracing off), the plugin can't attach to it. It prints a warning to stderr and exports nothing.