weightlab 0.1.0a1

Why WeightLab?

You fine-tuned a model. What changed inside its weights? Which matrices moved the most? Can you inspect individual parameter values without loading the whole checkpoint into memory?

WeightLab is a local-first, CPU-only toolkit for answering those questions. It reads SafeTensors checkpoints in place and offers a browser-based tensor explorer alongside a Python library and command-line checkpoint/LoRA change reports.

• See the structure: browse tensor metadata and explore entire matrices as multiresolution heatmaps.
• Get the actual numbers: select a region to inspect and copy lossless source values.
• Measure the change: rank checkpoint differences by tensor, layer, and role, with reproducible JSON, CSV, and Markdown reports.
• Control the cost: use bounded reads, cancellable pyramid builds, caching, and clearly labeled sampled previews.
• Keep weights local: no account, GPU, model inference runtime, or hosted backend required.

Scientific boundary: A change in weight magnitude does not demonstrate improved reasoning, accuracy, or behavior. WeightLab measures parameters; behavioral claims require independent evaluation and controlled experiments.

See it in action

These are real screenshots captured by browser tests on pinned Qwen2.5 checkpoints, not design mockups.

Whole-tensor heatmaps Pan, zoom, change aggregation, and select regions.	Exact region inspection Go from visual patterns to original tensor values.
Large embedding matrices Build an exact overview using resource-bounded pyramids.	Sampling you can see Previews disclose quality, row sampling, and coverage.

Workspace · Selecting a region · Pyramid build · ROI screenshots · More UI screenshots

What works today

Important distinction: You can compare checkpoints and supported LoRA adapters today via the CLI. The current UI visualizes individual checkpoint tensors, not a side-by-side checkpoint comparison.

Quickstart

Requirements: Python 3.12+, uv, and Git. No GPU or Node.js is needed to run the app. Repository CI covers macOS, Linux, and Windows; the real-checkpoint visual evaluation was run on macOS/Chromium.

git clone https://github.com/szaher/weightlab.git
cd weightlab
uv sync --locked
uv run weightlab start

Open the link printed in your terminal, which looks like:

http://127.0.0.1:<port>/#token=...

WeightLab serves the browser UI and local FastAPI backend on the same loopback origin. The URL contains a per-launch access token: open it locally, don't share it, and keep the terminal running.

Your first five minutes

1. Click Create project in the browser.
2. Import a checkpoint by absolute path, specifying an allowed selected root. You can use the repository's bundled examples/fixtures/tiny_base.safetensors and make examples/fixtures the selected root. This reads the file in place; there is no upload.
3. Open Explore, select a 2-D weight tensor, and inspect its shape, dtype, and metadata.
4. Open Overview to pan, zoom, change aggregation (mean, mean-absolute, RMS, min, max), and select a region.
5. Choose Inspect exact values to see the original source values in the Exact ROI heatmap or values table.

Print the fixture's absolute paths if needed:

uv run python -c "from pathlib import Path; p=Path('examples/fixtures').resolve(); print('Root:',p); print('Model:',p/'tiny_base.safetensors')"

No models to download: the bundled deterministic fixtures are enough to try the UI and CLI. To reproduce the real-Qwen workflow, see the pinned checkpoint validation.

Install a local wheel

uv build --wheel
uv tool install dist/weightlab-*.whl
weightlab start

The wheel includes the compiled web UI, so Node.js is only needed to rebuild it. WeightLab 0.1.0a1 (alpha) is being prepared for PyPI. After the first release is published, install with uv tool install "weightlab==0.1.0a1" or python -m pip install "weightlab==0.1.0a1". Until then, use the source or local wheel instructions above.

Compare checkpoints

Use weightlab diff to measure numerical changes between two checkpoints, including compatible sharded SafeTensors directories:

uv run weightlab diff \
  examples/fixtures/tiny_base.safetensors \
  examples/fixtures/tiny_ft.safetensors \
  --out ./weightlab-report

# report.json, report.md, tensors.csv, layers.csv, roles.csv

For your own checkpoints:

uv run weightlab diff /path/to/base /path/to/fine-tuned --out /path/to/reports

Or inspect a supported PEFT LoRA adapter's effective weight changes against its base:

uv run weightlab diff \
  examples/fixtures/tiny_base.safetensors \
  --adapter examples/fixtures/tiny_lora \
  --out ./weightlab-lora-report

Reports include per-tensor and rollup delta L2, delta RMS, relative L2, cosine similarity, max absolute delta, and non-finite counts. Compatibility classes distinguish verified file identity from declared lineage or mere shape matching. Support for LoRA variants is intentionally limited and unsupported modules are reported rather than silently interpreted.

Exit codes: 0 means complete; 3 means a report was written but is incomplete; 2 means error; 130 means cancelled. Check complete and any caveats before interpreting results.

Full details: CLI reference, metrics and LoRA support.

Python API

Read a bounded region from a checkpoint without loading the whole model:

from weightlab import open_checkpoint
from weightlab.analysis.roi import encode_roi

with open_checkpoint(
    "examples/fixtures/tiny_base.safetensors",
    selected_root="examples/fixtures",
) as checkpoint:
    print(checkpoint.metadata()[:3])
    roi = checkpoint.read_roi_raw("demo.layers.0.mlp.up_proj.weight", 0, 4, 0, 8)
    print(encode_roi(roi).to_json())

Compute a structured change report:

from weightlab.analysis.change_report import Options, run

report = run(
    "examples/fixtures/tiny_base.safetensors",
    "examples/fixtures/tiny_ft.safetensors",
    options=Options(sort="relative_l2"),
)
print(report["complete"])

The Python API remains pre-release. See change report documentation and reader implementation.

How it works

flowchart LR
    A["Local SafeTensors files"] --> B["Strict read-only reader"]
    B --> C["Catalog and project metadata"]
    B --> D["Bounded tile + pyramid engine"]
    D --> E["Whole-tensor heatmaps"]
    E --> F["Exact ROI drilldown"]
    B --> F
    B --> G["Streaming diff / LoRA analysis"]
    G --> H["JSON / CSV / Markdown reports"]

• Strict file reader: validates metadata, shards, offsets, and paths; reads only bounded file regions.
• Visual layer: multiresolution 256 × 256 tiles, optional exact pyramids, explicit row-sampled previews, and distinct coverage/precision labels.
• Numerical layer: raw ROI values are lossless; tile aggregate visualizations use float32 and must not be mistaken for exact individual parameters. Non-finite values and unrepresentable numbers get explicit states.
• Application: local FastAPI + React, SQLite for projects, bounded job handling and cache budgets.

Privacy and security

WeightLab runs locally by default and does not require a cloud account. The server accepts loopback-only binds and protects its local API with a launch token. Imported model files are read in place, not uploaded, rewritten, or deleted when a project is deleted. The application is designed without background telemetry or model-data exfiltration. Reports contain tensor names, shapes, statistics, file names, sizes, and possibly digests, so review reports before sharing. See security design.

Verified on real models

The project contains reproducible validation against pinned Qwen2.5-0.5B / Instruct and Qwen2.5-Math-1.5B / Instruct checkpoints. The recorded macOS run compared 1,256 tensor metadata entries, 184 exact ROI windows, and 1,048 tiles with an independent reader. It reported zero mismatches for the described numerical checks. During that run the backend's peak RSS was 166.3 MiB, with a 36 GiB RAM / Apple M3 Pro host; this is a benchmark observation, not a general memory guarantee or proof of the 8 GiB laptop target.

Reproduce the measurements and inspect limitations in real model evidence and the machine-readable benchmark. Browser validation was in Chromium; a practitioner usability study has not been completed.

Current scope and limitations

• Format: SafeTensors (single files or sharded indexes). No GGUF, PyTorch .bin, or general quantized checkpoint format support.
• Overview: whole-tensor heatmap aggregation is not a lossless display of every original scalar. Use Exact ROI for actual values.
• Comparison: numerical deltas are not causal evidence of improved behavior; independent checkpoints may be poorly aligned even when shapes match.
• UI: Compare/Adapters pages, graphical delta/LoRA overlays, stats jobs, and UI export are not built.
• Experimental features: inference, activations, transfer experiments, and evaluation are not present.
• Maturity: pre-release; field-tested practitioner feedback, non-Chromium browser coverage, and the intended 8 GiB laptop target remain unverified.

Development and contributing

The code is under active development. Use reproducible fixtures, add tests for numerical and security behavior, and avoid weakening validation to satisfy an acceptance gate.

uv sync --locked
uv run pytest
uv run ruff format --check .
uv run ruff check .
uv run mypy

Frontend changes require Node.js (see frontend documentation):

cd frontend
npm ci
npm run lint
npm run typecheck
npm test
npm run build
npx playwright install chromium
npm run e2e

Explore further: evaluation kit · test evidence · OpenAPI contract · architecture · numerical methods · roadmap · issue tracker.

For publishing and packaging instructions, see RELEASING.md.

The detailed version 1.2-final specification remains a frozen engineering baseline; it is not the current implementation status. Some historical planning documents and examples describe future behavior, so treat the tested source code and evidence above as the authority for currently working features.

License

No explicit repository-wide open-source license has been selected in the current source tree. Do not assume the code is licensed for reuse or redistribution until a license is added. Contributions and distribution should be discussed with the maintainer.