inflame

Where does your LLM inference latency live?

Feed it vLLM / TGI / Triton request traces; get a self-contained HTML report that shows, per request and fleet-wide, how wall time splits across tokenization → queue → prefill → decode → data io → network. Zero dependencies, stdlib only.

pip install inflame-trace

inflame report traces.jsonl -o report.html   # flamegraph-style HTML
inflame summary traces.jsonl                 # terminal table
inflame formats                              # how to produce each input

The report has three layers:

  1. Where latency lives — one icicle bar: share of total wall time per phase.
  2. Slowest requests — a waterfall of stacked per-request bars, phases in request order; hover any segment for exact timings, hover the legend to isolate a phase.
  3. Phase percentiles — p50/p95/p99 per phase, as a table.

The terminal summary gives you the same phase split as a bar chart you can paste into an incident channel.

Getting traces out of your stack

Stack How Phases you get
vLLM vllm serve MODEL --otlp-traces-endpoint http://collector:4318/v1/traces, collector writes a JSON/file export queue, prefill, decode
TGI text-generation-launcher --json-output ... 2>&1 | tee tgi.log tokenization, queue, inference
Triton tritonserver --trace-config mode=triton --trace-config triton,file=trace.json --trace-config rate=1 --trace-config level=TIMESTAMPS network, queue, data io, inference

vLLM inputs are accepted as raw OTLP exports, JSON arrays, or JSONL of spans — whatever your collector writes. Format is auto-detected; force it with --format vllm|tgi|triton.

Reading the report

  • queue dominating → batch scheduler is saturated: add replicas, cap concurrency upstream, or shrink max batch tokens.
  • prefill dominating → long prompts: cache system prompts (prefix caching), trim context, or split prefill across GPUs.
  • decode dominating → normal for long generations; compare decode tok/s against your quantization/kernel expectations — a drop here is where quantization overhead or a mis-tuned kernel shows up.
  • data io / network visible at all → payloads are too fat (send token ids, not logits; compress embeddings).

Honest limitations

  • TGI does not expose a prefill/decode split; its model compute is one inference phase (completion tokens are estimated from time_per_token).
  • vLLM's periodic Avg prompt throughput stats log has no per-request data and is rejected with a pointer to the OTLP setup.
  • Triton phases describe the server's view (queue/compute/marshalling); for in-model phase splits use an ensemble that traces per step.

Demo

python3 scripts/make_demo.py   # deterministic 240-request sample -> demo/index.html

Dev

python3 -m pytest -q   # 60+ tests, no network, no deps

MIT.

Project links

Data verified by PyPI on Aug 10, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for sophie.nguyenthuthuy from gravatar.com sophie.nguyenthuthuy

Credits

Author: Sophie Nguyen

GitHub Statistics

Data verified by PyPI on Aug 10, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.9

Tags

llm inference latency vllm tgi triton flamegraph observability

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

inflame_trace-0.1.0.tar.gz (22.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

inflame_trace-0.1.0-py3-none-any.whl (20.1 kB view details)

Uploaded Python 3

File details

Details for the file inflame_trace-0.1.0.tar.gz.

File metadata

  • Download URL: inflame_trace-0.1.0.tar.gz
  • Upload date:
  • Size: 22.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for inflame_trace-0.1.0.tar.gz
Algorithm Hash digest
SHA256 d586bd5b238d1b4e924aec7634c9ace65c48e535ce7081af81d6ac7666667cf0
MD5 707424aa2eeac95d1e8e3db3b6800fb6
BLAKE2b-256 6c92370adc2d014f4dff95f389851b940e57f0bae9b74c2bdffb692012dbefd9

See more details on using hashes here.

Provenance

The following attestation bundles were made for inflame_trace-0.1.0.tar.gz:

Publisher: publish.yml on sophie-nguyenthuthuy/inflame

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file inflame_trace-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: inflame_trace-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 20.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for inflame_trace-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 49142043f163566eb58f810f2011888a4e1f05958087b3973219e02d03415c5e
MD5 57fa2f2b066ea8019b63a3d3bdca5f62
BLAKE2b-256 3a6b2b45ce604628ce0ec9d46560cf948ad937eff9c2908e05eb069342937692

See more details on using hashes here.

Provenance

The following attestation bundles were made for inflame_trace-0.1.0-py3-none-any.whl:

Publisher: publish.yml on sophie-nguyenthuthuy/inflame

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page