safetypulse 0.1.1

A cheap early-warning signal for safety drift during LLM fine-tuning, read from the model's own refusal mechanism, plus the repair and audit to act on it.

https://github.com/user-attachments/assets/2f55f4b7-e20f-4043-a141-77cbba8fa475

Article · Report · Quick start · Reproduce · Responsible release

The problem

Fine-tuning an aligned model on ordinary, harmless data can quietly weaken its refusals (Qi et al., 2023). Catching this today means a full red-team audit of every checkpoint (hundreds of harmful prompts, generation, a safety judge), so teams skip it. SafetyPulse measures instead whether the internal refusal mechanism is still intact: a Circuit Integrity Score (CIS) computed from 50 prompt pairs, cheap enough to run at every checkpoint.

Results (held-out test runs; full report: docs/report/REPORT.md)

How the four toolkits are chained

CuratorKIT ─► LoRA fine-tune ─► each checkpoint ─► safetypulse check (CIS) ─► PASS: continue
 (clean data)                                              │
                                                           └─ WARN/FAIL ─► SafeTune repair ─► AuditKIT before/after
CircuitKIT: independent check of the refusal circuit that CIS measures

Hugging Face artifacts

All under huggingface.co/purvansh01:

Withheld on purpose: the positive-control adapters trained on harmful data and the text of harmful model answers. Why, and how judges can get access: docs/11_WITHHELD_ARTIFACTS.md.

Install and use

Python 3.12 (SafeTune needs it; check alone runs on 3.10+), a GPU with 16 GB or more, and access to the gated Tiny Aya Global (accept its terms, then hf auth login).

pip install "safetypulse[peft,toolkits]"   # toolkits = CircuitKIT, SafeTune, AuditKIT; "safetypulse[peft]" is enough for check

# Probe bundle and one drifted checkpoint (alpaca-s1, step 125)
hf download purvansh01/safetypulse-probe-bundle --repo-type dataset --local-dir bundle --exclude "unpaired_v1/*"
hf download purvansh01/safetypulse-lora-alpaca --include "alpaca-s1/checkpoint-125/*" --local-dir lora

# Score it: exit code 0 PASS, 10 WARN, 20 FAIL (this one FAILs, cis_circ ≈ 0.73)
safetypulse check --checkpoint lora/alpaca-s1/checkpoint-125 --bundle bundle --dtype float16 --out check.json

# Repair it with SafeTune's safety-vector restore (settings chosen on dev), then check again (PASS, ≈ 0.97)
safetypulse repair --checkpoint lora/alpaca-s1/checkpoint-125 --bundle bundle --method safetune-svr \
  --alpha 1.0 --svr-rank 8 --dtype float16 --out repaired
safetypulse check --checkpoint repaired --bundle bundle --dtype float16

# Audit: ASR, over-refusal, GSM8K, MMLU with bootstrap CIs (Llama Guard 3 1B is gated too)
hf download purvansh01/safetypulse-eval --repo-type dataset --local-dir eval
safetypulse audit --model repaired --data-dir eval --split test --judge llamaguard \
  --judge-model meta-llama/Llama-Guard-3-1B --out audit.json

The thresholds were calibrated with the model in fp16, so score in float16 for numbers comparable to ours.

Inside a Hugging Face Trainer:

from safetypulse.integrations.hf_callback import SafetyPulseCallback
trainer.add_callback(SafetyPulseCallback("bundle", on_fail="stop"))   # checks every saved checkpoint

safetypulse calibrate builds a bundle for a new base model from contrast pairs (--pairs), and --circuitkit-crosscheck adds the CircuitKIT comparison.

Reproducibility

Every experiment is a notebook in kaggle/notebooks/, generated from tools/build_notebooks.py and run on Kaggle (T4 GPUs) with tools/kaggle_run.py. Each run's executed notebook, log, key output files and a RUN_SUMMARY.md (what ran, numbers, interpretation) are in kaggle/executed/.

Fixed choices: seed 0 for data splits; fine-tune seeds 0 (dev) and 1–3 (test); dev and test runs never overlap; thresholds and repair strengths are chosen on dev only, then applied unchanged to test. Model and judge revisions and dataset hashes are recorded in kaggle/executed/*/RUN_SUMMARY.md and safetypulse-eval/manifest.json. Package tests, from a clone: pip install -e ".[dev,peft]" && pytest (186 tests, run in CI on Python 3.10–3.12).

Documentation

Report · Research plan · Architecture · Data · Evaluation protocol · Risk, ethics, licensing · Withheld artifacts · Kaggle and HF resources · Decisions

Responsible use

CIS is a screening signal: PASS does not certify a model as safe, and release decisions need a full audit. Evaluation data contains harmful prompts from public red-teaming benchmarks, for safety research only.

Licence

Code: Apache-2.0 (ADR-0004). Released adapters and data: CC-BY-NC-4.0, inherited from Tiny Aya and Alpaca; benchmark data keeps its source licence. The Lexsi toolkits keep their own licences and are used only through safetypulse.adapters.