weightscope 0.1.0

X-ray for AI model weights — sparse tensor diff for any two HuggingFace models.

Weightscope

X-ray for AI model weights.

See exactly what changed between any two models — by the numbers, not the marketing copy.

CLI + Python library + Gradio HuggingFace Space. Live at weightscope.dev.

What it does

Every fine-tune ships as a multi-gigabyte blob and a vague changelog. Weightscope diffs the actual weights between any two HuggingFace model checkpoints and shows you which layers changed, by how much, and how sparse the update really is — typically 99% of weights don't move. Paste two HuggingFace model IDs for a free public report, or run it privately on your own checkpoints.

It streams .safetensors files tensor-by-tensor, so it does not load either full model into memory.

You get:

• Which tensors changed, by how much
• Layer-level rollup (most-changed layer groups)
• Estimated sparse-delta size vs full-model size — the compression you'd get if you only shipped the diff instead of the full model
• Tensors only in A or only in B (architectural changes)

Why this exists

When someone uploads Llama-3-70B-Medical-v2 to HuggingFace, there is no way to see what actually changed vs the base model. You'd have to download both 140 GB blobs and manually compare. As a result:

• Model laundering — rebranded copies are uploaded as "new" models.
• No quality signal — a LoRA that touched 0.1% of weights vs a full fine-tune that touched 15% tell you very different things about the model. Currently invisible.
• Wasted bandwidth — HuggingFace ships petabytes/day of redundant weights because there's no deduplication.

Weightscope is the missing git diff for model weights.

Install

pip install weightscope

Or from source:

git clone https://github.com/weightscope/weightscope
pip install -e weightscope

(PyPI release: TODO)

Usage

weightscope Qwen/Qwen2.5-0.5B Qwen/Qwen2.5-0.5B-Instruct
weightscope -o report.md meta-llama/Llama-3.2-1B meta-llama/Llama-3.2-1B-Instruct
weightscope --threshold 1e-4 model_a model_b

Output (abbreviated):

# Model diff: Qwen/Qwen2.5-0.5B vs Qwen/Qwen2.5-0.5B-Instruct
- Total params compared: 494,032,768
- Changed params: 487,191,488 (98.62%)
- Sparsity (unchanged fraction): 1.38%
- Full model size: 988.1 MB
- Sparse delta size (estimated): 2.78 GB
- Compression ratio: 0.3x

(In this case the Instruct model is essentially a full retrain — the delta is bigger than the model. Compare to a LoRA fine-tune to see the wins: typically 99.9% sparsity, 1000x compression.)

Library use

from weightscope import diff_models, render_markdown

md = diff_models("Qwen/Qwen2.5-0.5B", "Qwen/Qwen2.5-0.5B-Instruct")
print(render_markdown(md))

print(f"Sparsity: {md.overall_sparsity:.2%}")
print(f"Compressed delta: {md.compressed_delta_bytes / 1e9:.2f} GB")
for t in md.tensors[:5]:
    print(t.name, t.sparsity)

HuggingFace Space

The Gradio app in app.py runs on a free HF Space. Free Spaces have ~16 GB RAM and CPU only, so this works for total-model-size up to about 2 GB. For larger models, run the CLI locally.

Pricing

The CLI and library are MIT-licensed and free forever — including for commercial use. Run as much as you want on your own machines.

The hosted product at weightscope.dev adds:

Plan	Price	What you get
Free	$0	Public diffs, the CLI, the HuggingFace Space
Pro	$9/mo	Private diffs on your own checkpoints, hosted history, shareable links
Team	$39/mo	Up to 5 seats, shared workspace, audit log, SSO
Enterprise	Custom	On-prem, signed compliance reports, custom SLAs

If your subscription lapses (card declines, cancellation, etc.) we never delete your data and never lock the CLI. You drop back to Free-tier limits for new artifacts; everything you already created stays accessible.

Roadmap

• v0.1 (this MVP): CLI, library, Gradio Space. Sparse diff + markdown report.
• v0.2: Compressed delta serialization (weightscope export A B delta.ws) so the delta file can actually be downloaded and applied.
• v0.3: Lineage graph — given a model B, search a public registry for the model A it most likely descends from.
• v0.4: GitHub Action that auto-runs on model PRs.
• v1.0: Hosted registry for private model deltas (paid product).
• Quantization-aware diff — compare fp16 to int8 quants of the same model.

Status

MVP scaffold. Tested on small open models (Qwen 0.5B, GPT-2 family). Not yet packaged on PyPI.

License

MIT.