weight-audit

Scan the open-weight AI models a project depends on (Llama, Gemma, Qwen, DeepSeek, and others) for license compliance risk before a scale-cap or acceptable-use restriction turns into a legal problem for a commercial product.

Why

Open-weight models are not all "open source." The Llama Community License caps free commercial use at 700M monthly active users. Gemma, Qwen, and DeepSeek carry acceptable-use policies. CC-BY-NC weights forbid commercial use outright. Roughly 70% of models on Hugging Face ship with no license metadata at all. Enterprise SCA platforms (Endor Labs, Sonatype, Black Duck) now gate on this for large customers -- this tool does the same check for the developers and small AI startups those platforms don't sell to.

Install

pip install weight-audit

Usage

weight-audit scan .                        # scan a directory (manifest + source detection)
weight-audit scan . --policy policy.json    # use a custom policy
weight-audit scan . --json                  # machine-readable output for CI
weight-audit scan . --online                # also query the HF Hub API for models not in the offline DB
weight-audit bom . --output ml-bom.json     # generate a CycloneDX ML-BOM
weight-audit report . --license-key <KEY> -o evidence.html   # [Pro] compliance evidence report

Exit code is 1 if any model violates the policy, 0 otherwise -- safe to drop into CI:

- run: weight-audit scan .

Detecting models

weight-audit looks for models in two places:

1. A weight-audit-models.txt manifest in the scanned directory -- one Hugging Face model id per line, with an optional declared license as a second column:

   meta-llama/Llama-3-8B-Instruct
   some-org/attributed-model,cc-by-4.0
   

2. A best-effort scan of .py source for AutoModel.from_pretrained("org/model") and model_id = "org/model" patterns. This is a convenience for a first scan -- for reliable CI gating, use a manifest.

Policy

By default, anything at strong-restriction or above is a violation, and models with no license metadata at all (unknown) are treated as violations too. Override with a JSON file:

{
  "fail_at_or_above": "weak-restriction",
  "treat_unknown_as_violation": false
}

How it classifies

Licenses are bucketed into four tiers: permissive < weak-restriction < strong-restriction < unknown. 27 named model families (Llama, Gemma, Qwen, DeepSeek, BLOOM, StarCoder, Baichuan, ChatGLM, Yi, Grok, FLUX, and others) are matched against a hand-curated offline database recording their actual restriction -- not just the license name -- so a scale cap or acceptable-use policy shows up as a concrete obligation, not just a label. See weight_audit/license_db.py for the exact rules and weight_audit/classify.py for the resolution order.

A broader SPDX/common-identifier fallback table (Apache-2.0, MIT, CC-BY variants, OpenRAIL variants, and named Llama/Gemma license strings) covers models outside the named-family list when a license string is available, either from a manifest's declared_license column or from --online.

Online mode

--online queries the Hugging Face Hub API for models that aren't in the offline database and don't have a declared license in the manifest. It never overrides a license already recorded in the manifest, and any network failure (timeout, 404, malformed response) degrades to unknown rather than crashing the scan -- the offline database remains the reliable default; --online only fills gaps.

Limitations

The offline database is a hand-curated table of well-known model families, not a registry mirror. Models outside the known families and without a declared license (from a manifest or --online) are flagged unknown. This is deliberate: "unknown" surfaces the real state of the ecosystem (~70% of Hugging Face models carry no license) rather than silently passing them.

Pro

weight-audit report generates a standalone HTML compliance-evidence document -- per-model tier, concrete obligation, and violation status, framed as supporting evidence for EU AI Act Annex IV / Annex XI GPAI technical documentation and EU CRA record-keeping. scan and bom stay free and unrestricted; only this evidence-export report requires a one-time license key ($99):

weight-audit report . --license-key <KEY> -o evidence.html

Unlock it here: https://buy.stripe.com/PLACEHOLDER_WEIGHT_AUDIT_PRO (the key can also be set via the WEIGHT_AUDIT_KEY environment variable instead of --license-key).

Built for AI startups and individual developers who need EU AI Act/CRA-ready evidence without buying into an enterprise SCA platform (Endor Labs, Sonatype, Black Duck) built for a much larger buyer.

License

MIT