llmlatency

Measured latency and uptime for AI inference APIs, as a small Python client for the open dataset published at llmlatency.dev.

Probes run every five minutes from four regions and are never routed through a gateway or an aggregator, so the numbers describe the providers themselves rather than a proxy in front of them.

• Network probe — DNS → TCP → TLS → time to first byte.
• Inference probe — time to first token on a real completion request.

These are different quantities, an order of magnitude apart, and this library never mixes them in one ranking.

Install

pip install llmlatency

No dependencies — standard library only.

Use it from Python

import llmlatency

llmlatency.regions()
# ['ap-tokyo', 'eu-hetzner', 'sa-east', 'us-central']

llmlatency.fastest("eu-hetzner")
# {'rank': 1, 'provider': 'nscale', 'p50_ms': 99, 'p95_ms': 203,
#  'uptime_pct': 100, 'samples': 289}

for row in llmlatency.ranking("us-central")[:3]:
    print(row["rank"], row["provider"], row["p50_ms"], "ms")

llmlatency.provider("anthropic")          # one provider across every region
llmlatency.generated_at()                 # snapshot timestamp, ISO-8601 UTC

Fetch once and pass the snapshot around if you make several queries:

data = llmlatency.fetch()
fast = {r: llmlatency.fastest(r, data=data) for r in llmlatency.regions(data)}

Use it from the shell

llmlatency fastest                        # fastest provider in every region
llmlatency ranking --region eu-hetzner --top 5
llmlatency provider openai
llmlatency fastest --probe inference --json

Data, methodology, licence

• Live rankings and full methodology: https://llmlatency.dev
• Machine-readable snapshot: https://llmlatency.dev/api/rankings.json
• Citable archive with a DOI: https://doi.org/10.5281/zenodo.21954788
• Daily snapshots in git: https://github.com/mazamaka/llm-latency-tracker

The dataset is CC-BY-4.0; this client library is MIT.

Limitations, stated up front. Vantage points are cloud data centres, not consumer networks, so absolute values are lower than an end user would see — the comparison between providers is the meaningful part. Provider coverage changes over time as APIs appear and shut down.