weightscope 0.2.0

X-ray for AI model weights — sparse tensor diff + delta-as-patch (.wsp) for any two HuggingFace models.

Weightscope

X-ray for AI model weights.

See exactly what changed between any two models — by the numbers, not the marketing copy.

CLI + Python library + Gradio HuggingFace Space. Live at weightscope.dev.

What it does

Every fine-tune ships as a multi-gigabyte blob and a vague changelog. Weightscope diffs the actual weights between any two HuggingFace model checkpoints and shows you which layers changed, by how much, and how sparse the update really is — typically 99% of weights don't move. Paste two HuggingFace model IDs for a free public report, or run it privately on your own checkpoints.

It streams .safetensors files tensor-by-tensor, so it does not load either full model into memory.

You get:

• Which tensors changed, by how much
• Layer-level rollup (most-changed layer groups)
• Estimated sparse-delta size vs full-model size — the compression you'd get if you only shipped the diff instead of the full model
• Tensors only in A or only in B (architectural changes)

Why this exists

When someone uploads Llama-3-70B-Medical-v2 to HuggingFace, there is no way to see what actually changed vs the base model. You'd have to download both 140 GB blobs and manually compare. As a result:

• Model laundering — rebranded copies are uploaded as "new" models.
• No quality signal — a LoRA that touched 0.1% of weights vs a full fine-tune that touched 15% tell you very different things about the model. Currently invisible.
• Wasted bandwidth — HuggingFace ships petabytes/day of redundant weights because there's no deduplication.

Weightscope is the missing git diff for model weights.

Install

pip install weightscope

Or from source:

git clone https://github.com/weightscope/weightscope
pip install -e weightscope

Usage

Three subcommands: diff, export, apply.

# Markdown report of what changed between A and B
weightscope diff Qwen/Qwen2.5-0.5B Qwen/Qwen2.5-0.5B-Instruct
weightscope diff -o report.md meta-llama/Llama-3.2-1B meta-llama/Llama-3.2-1B-Instruct
weightscope diff --threshold 1e-4 model_a model_b

# Pack the A→B delta as a single sparse patch file (.wsp)
weightscope export Qwen/Qwen2.5-0.5B Qwen/Qwen2.5-0.5B-Instruct -o qwen.wsp

# Apply that patch to the base, producing a consolidated .safetensors
weightscope apply qwen.wsp Qwen/Qwen2.5-0.5B -o qwen_instruct.safetensors

apply verifies a sha256 fingerprint of each base tensor before touching it, so a patch can only be applied to the exact checkpoint it was built against. Pass --no-verify to force.

Output (abbreviated):

# Model diff: Qwen/Qwen2.5-0.5B vs Qwen/Qwen2.5-0.5B-Instruct
- Total params compared: 494,032,768
- Changed params: 487,191,488 (98.62%)
- Sparsity (unchanged fraction): 1.38%
- Full model size: 988.1 MB
- Sparse delta size (estimated): 2.78 GB
- Compression ratio: 0.3x

(In this case the Instruct model is essentially a full retrain — the delta is bigger than the model. Compare to a LoRA fine-tune to see the wins: typically 99.9% sparsity, 1000x compression.)

Library use

from pathlib import Path
from weightscope import diff_models, render_markdown, export_patch, apply_patch

# Diff report
md = diff_models("Qwen/Qwen2.5-0.5B", "Qwen/Qwen2.5-0.5B-Instruct")
print(render_markdown(md))

print(f"Sparsity: {md.overall_sparsity:.2%}")
print(f"Compressed delta: {md.compressed_delta_bytes / 1e9:.2f} GB")
for t in md.tensors[:5]:
    print(t.name, t.sparsity)

# Build and apply a sparse patch
export_patch("Qwen/Qwen2.5-0.5B", "Qwen/Qwen2.5-0.5B-Instruct",
             out_path=Path("qwen.wsp"))
apply_patch(patch_path=Path("qwen.wsp"),
            base="Qwen/Qwen2.5-0.5B",
            out_path=Path("qwen_instruct.safetensors"))

HuggingFace Space

The Gradio app in app.py runs on a free HF Space. Free Spaces have ~16 GB RAM and CPU only, so this works for total-model-size up to about 2 GB. For larger models, run the CLI locally.

Pricing

The CLI and library are MIT-licensed and free forever — including for commercial use. Run as much as you want on your own machines.

The hosted product at weightscope.dev adds:

Plan	Price	What you get
Free	$0	Public diffs, the CLI, the HuggingFace Space
Pro	$9/mo	Private diffs on your own checkpoints, hosted history, shareable links
Team	$39/mo	Up to 5 seats, shared workspace, audit log, SSO
Enterprise	Custom	On-prem, signed compliance reports, custom SLAs

If your subscription lapses (card declines, cancellation, etc.) we never delete your data and never lock the CLI. You drop back to Free-tier limits for new artifacts; everything you already created stays accessible.

The .wsp patch format

A weightscope patch is a single self-describing file:

4-byte magic "WSP2" | 4-byte LE header length | UTF-8 JSON header | zstd(blobs)

The JSON header lists every tensor with {name, shape, dtype, kind, blob_offset, blob_size}. A delta kind additionally carries n_changed, index_width, and base_sha256. The zstd stream is the concatenation of each tensor's blob: for a delta, [changed indices][raw B-values at those indices]; for a new tensor, the full raw bytes. Tensors present only in A are listed in only_in_a and get removed on apply.

Because the header is plain JSON, you can inspect a patch with:

from weightscope.patch import read_patch
with open("qwen.wsp", "rb") as fp:
    p = read_patch(fp)
for r in p.records[:5]:
    print(r.name, r.shape, r.kind, r.n_changed)

Roadmap

• v0.1: CLI, library, Gradio Space. Sparse diff + markdown report. ✅
• v0.2 (current): weightscope export / apply — sparse delta patch file (.wsp) that round-trips to the target model bit-exactly, with sha256 base verification. ✅
• v0.3: Plain-English interpretation in the report — separate Facts (verifiable) from Inferences (heuristic).
• v0.4: Lineage graph — given a model B, search a public registry for the model A it most likely descends from.
• v0.5: GitHub Action that auto-runs on model PRs.
• v1.0: Hosted registry for private model deltas (paid product).
• Quantization-aware diff — compare fp16 to int8 quants of the same model.

Status

v0.2 — diff, export, and apply work on small open models (Qwen 0.5B, GPT-2 family) and on synthetic fixtures with bit-exact round-trip verified.

License

MIT.