memaudit

Training-data memorization auditor for Hugging Face Trainer / TRL fine-tunes.

A local, Apache-2.0 plugin that answers two questions every fine-tune in a regulated setting should document (EDPB Opinion 28/2024 para 55 / para 58):

1. Membership -- can an attacker with logprob access tell what was trained on?
2. Regurgitation -- does the model emit training content when prompted with a prefix?

memaudit injects pre-registered canaries into the raw dataset, runs a PEFT-aware pre-flight when training starts, and writes memaudit-report.json when training ends. The same engine is available post-hoc (memaudit audit --ref auto). Entirely on your machine -- no phone-home, no account, no SaaS.

This tool produces evidence of resistance to the attacks it actually runs. It does not make you GDPR / AI Act / CNIL compliant.

Install

pip install memaudit                 # core
pip install "memaudit[peft,trl]"     # LoRA + SFTTrainer lint (recommended for fine-tunes)
# from source:
git clone https://github.com/mem-audit/memaudit.git && cd memaudit
pip install -e ".[dev,peft,trl]"

Requires Python 3.10+ and transformers>=4.56.2 (works on 5.x).

30-second start

pip install memaudit
memaudit demo

Positive-control demo on a tiny overfit model -- proves the instrument can detect memorization and keep controls clean. Checked-in report: examples/demo-report.json.

Usage

from memaudit import generate_canaries, inject, MemorizationAuditCallback

canaries = generate_canaries(
    tokenizer, n=32, n_controls=100, family="high_ppl",
    repetitions=(1, 4, 16), seed=0,
)
train_ds, manifest = inject(train_ds, canaries, fmt="auto", seed=0)

trainer.add_callback(
    MemorizationAuditCallback(
        trainer=trainer, manifest=manifest, real_sample=64, ref="auto",
    )
)
trainer.train()   # writes <output_dir>/memaudit-report.json

Injection is a pre-train helper -- transformers builds the dataloader before on_train_begin, and TRL tokenizes inside SFTTrainer.__init__. The secret goes on the trainable side (completion / assistant turn / text body); prompt-side placement is refused.

Post-hoc (or after ZeRO-3 / FSDP, where in-callback scoring is deferred):

memaudit audit --model ./out --canary-set ./out/memaudit-manifest.json \
               --dataset ./train.jsonl --ref auto

ref="auto" uses peft.disable_adapter() on an unmerged LoRA (one model copy). Full fine-tunes need --ref <base-checkpoint> or explicit --ref none.

What you get

Every run writes schema 1.2.0 JSON with both verdicts, negative controls, Clopper-Pearson CIs, provenance hashes, a shipped EDPB compliance_annex, and an explicit limitations statement.

Membership and regurgitation routinely disagree. Scores are on the secret span only; full-sequence loss collapses detection.

Also in the report: membership.by_repetition (1x / 4x / 16x / pooled), membership.auc (secondary), canaries.requested_family / actual_generator, negative_controls, optional real_records.set_level (needs held_out=), compliance_annex, report_sha256 + sidecar, phone_home: false.

Pluggable membership scorers: docs/membership-scorers.md.

Measured results

Flagship: TinyLlama + Alpaca LoRA (powered)

Scale: TinyLlama-1.1B-Chat, 20,000 tatsu-lab/alpaca rows, LoRA r=8, 1 epoch, 100 inserted / 200 controls, 0.874% canary budget. Measured 2026-08-30 (Apple M3 Pro, MPS). Full write-up: docs/case-study-alpaca.md | live site.

The pooled 10% headline is substantially a duplication/exposure stress signal (16x tier drives most detections), not a 10% probability for a single-exposure record. Regurgitation is under this prefix/decoding/exact-match protocol only.

Reproduce: pip install "memaudit[peft,trl]" && python examples/alpaca_case_study.py

Other measured runs

A 12-canary Alpaca first look (TPR 0.500, wide CI) is archived in examples/alpaca-case-study-report.json -- not a headline. See the case-study appendix.

Profiles & CLI

memaudit report out/memaudit-report.json          # EDPB annex as markdown
memaudit verify out/memaudit-report.json          # integrity check (content hash)
memaudit doctor --output-dir examples             # env + demo + schema
memaudit audit ... --seeds 0,1,2                  # audit-procedure stability block
memaudit audit ... --release-context public-api   # para 46 release context
bash scripts/acceptance.sh                          # buyer acceptance gate

Public helper: from memaudit import inject (implementation: memaudit.injection).

Scope

Default canaries use the existing vocabulary -- memaudit never resizes the vocab. Pre-flight blocks silent false confidence (wrong placement, masked labels, empty inclusion coins, <100 controls for headline TPR). TPR@1%FPR is refused, not fabricated, when underpowered.

Architecture

generate_canaries() -> inject() -> MemorizationAuditCallback -> run_audit() / memaudit audit

Injection cannot live in the callback (dataloader + TRL tokenization happen first). ZeRO-3/FSDP defer in-callback scoring to the CLI. Ten implementation landmines (no callback injection, secret-span scoring, processing_class, two verdicts always) are encoded in source -- see module docstrings.

Limitations

• Small canary counts -> wide CIs. v0.1 defaults are a CPU-friendly start, not a regulatory sample size.
• Thresholds calibrate on this run's controls; they do not transfer across model families.
• Real-record ranking without held_out= is descriptive only -- no FPR, not evidence about individuals.
• Black-box final-model audits are structurally loose. A small TPR is not a privacy certificate.
• Multi-seed mode measures audit-procedure variance (bootstrap calibration + real-record sampling), not re-training variance.
• DPO / GRPO / Hub model-card push out of scope for v0.1. LoRA-aware, not LoRA-only.

Verified stack

Known-bad: transformers 5.16.x + torch 2.6.dev hangs on FSDP imports. Recommended LoRA pin: transformers==4.56.2, torch>=2.5,<2.8, peft==0.20.0. Do not use --system-site-packages over a conda torch nightly.

Links