fraQtl Diagnostic

Fingerprint any transformer's compression potential — fast.

For public installs, use fraqtl-diagnostic >= 0.2.0 for the inference-readiness scanner.

Measures per-layer:

• γ (stretched-exponential decay shape of the Hessian spectrum)
• knee (spectrum cutoff index)
• k95 (directions needed for 95% of eigenvalue energy)
• depth-law (how decay shape evolves across layers)
• compression potential + suggested bit budgets (Shannon-based)

Works on any HuggingFace-compatible transformer. ~3 min for a 0.5B model on A100, ~5 min for 1B, ~10 min for 7B.

Install

pip install fraqtl-diagnostic           # >=0.2.0 on PyPI
pip install -e /path/to/diagnostic-public  # editable install from source

Use

fraqtl analyze meta-llama/Llama-3.2-1B-Instruct

Inference readiness scan

diagnose-inference checks whether a model's config appears ready for a target serving context. It uses HuggingFace config.json plus textbook memory math: no GPU, no model load, no compression run.

fraqtl diagnose-inference Qwen/Qwen2.5-7B-Instruct --context 65536

What you see in the terminal:

fraQtl Model Context Receipt
────────────────────────────

MODEL
  model:      Qwen/Qwen2.5-7B-Instruct
  arch:       qwen2
  layers:     28
  heads:      28
  kv_heads:   4
  attention:  GQA

CONTEXT
  original RoPE context:  32,768
  requested context:      65,536
  status:                 [!] exceeds original context
  long-context scaling:   [!] YaRN required, not configured

KV CACHE ESTIMATE
  fp16 KV @ 8K:       0.47 GB
  fp16 KV @ 16K:      0.94 GB
  fp16 KV @ 32K:      1.88 GB
  fp16 KV @ 64K:      3.76 GB
  fp16 KV @ 128K:     7.52 GB

READINESS FLAGS
  [!]  CONTEXT_EXCEEDS_NATIVE
  [!]  ROPE_SCALING_REQUIRED
  [!]  YARN_REQUIRED
  [!]  YARN_MISSING
  [!]  NEEDS_PEAK_VRAM_MEASUREMENT
  [!]  NEEDS_RETRIEVAL_TEST

NEXT TESTS
  [ ] perplexity
  [ ] needle-in-a-haystack / passkey retrieval
  [ ] decode toks/sec
  [ ] prefill toks/sec
  [ ] peak VRAM
  [ ] batch concurrency

VERDICT
  Not a benchmark yet.
  This is the receipt.

The command also writes a *_diagnose-inference_*.json machine-readable receipt to disk for downstream pipelines.

For gated models (Llama, Gemma, etc.) pass an HF token:

HF_TOKEN=$(cat ~/.cache/huggingface/token) \
  fraqtl diagnose-inference meta-llama/Llama-3.1-8B-Instruct --context 65536

Or set HF_TOKEN / HUGGINGFACE_HUB_TOKEN in your environment.

Don't want to install anything? Run it in your browser at huggingface.co/spaces/fraQtl/fraqtl-diagnostic.

from fraqtl_diagnostic import analyze

report = analyze("meta-llama/Llama-3.2-1B-Instruct")
print(report.summary())
report.to_html("llama-1b_fingerprint.html")
report.to_png("llama-1b_fingerprint.png")

Try it on your GPU in one command (Modal)

If you don't want to fight Python-env dependencies locally, the fastest way to try the tool on a real model is via Modal (free tier gives you an A100):

# one-time: `pip install modal && modal setup`
# assumes a Modal secret named `huggingface` with an HF token

cd diagnostic-public/
modal run tests/modal_try.py --model-id Qwen/Qwen2.5-0.5B
modal run tests/modal_try.py --model-id TinyLlama/TinyLlama-1.1B-Chat-v1.0
modal run tests/modal_try.py --model-id mistralai/Mistral-7B-v0.1 --n-seqs 32 --seq-len 512

# pull the report back:
modal volume get fraqtl-hf-cache fraqtl-results/diagnostic-smoke ./reports/

What you get

Three outputs, same data, different framings:

• *.json — machine-readable per-layer fingerprint (feed into other tools)
• *.html — human-readable report with tables + embedded figure
• *.png — 4-panel figure: spectrum overlay, γ depth-law, k95/layer, summary

How to read the output

γ (stretched-exponential shape parameter)

The Hessian input-covariance spectrum λ_i of each linear projection is fit against λ_i ≈ exp(−β · i^γ + c). γ is the shape of the decay:

Lower γ = more compression headroom.

k95 / dim

Fraction of eigendirections needed to capture 95% of eigenvalue energy. A value of 0.1 means "95% of the Hessian mass lives in the top 10% of directions" — prime territory for rank-preserving compression. Values typical on production transformers:

Depth-law

Linear fit of γ across layer depth. A negative slope is the common case (shallow layers exponential, deep layers more stretched). The magnitude of the slope × R² tells you whether the shape is a stable architecture property or noisy per-layer.

Suggested bit budget

Shannon-derived bits-per-weight that the information-theoretic ceiling can tolerate at three conservatism tiers. This is a ceiling, not a prediction. Real PPL loss from compression depends on the implementation. The diagnostic tells you how much room the math leaves; the actual compression run tells you how close to the ceiling you got.

Status

v0.2 (current): diagnostic metrics + suggested bit budgets, plus diagnose-inference for config-level serving-readiness checks.

v1.0 (coming with Paper 3, ~4 weeks): adds Shannon-efficiency grading — "your model is at X% of the theoretical ceiling vs competitors at Y%."

Same pip install fraqtl-diagnostic — grading is a layer on top of the existing diagnostic, not a separate tool.

How it works (one-paragraph summary)

For each target projection W : ℝ^d_in → ℝ^d_out, we capture the input covariance H = E[x^T x] on wikitext-2 calibration, then eigendecompose it. The spectrum λ_i encodes how much of the layer's Jacobian mass lives along each eigendirection of the input distribution. Tight universal shape (fixed γ across layers) implies compressible redundancy; a fat-tailed spectrum (high k95/dim) implies less. Shannon rate-distortion gives the information-theoretic floor D*(R) = geomean(λ) · 2^(−2R) at any bit budget R, which the diagnostic reports.

Full derivation + universality data across 8 architectures is in the forthcoming Paper 3.

Want to actually compress your model?

The diagnostic tells you the ceiling. The compression engine is the closed part of the product:

fraqtl.ai/compress

License

Apache 2.0.