pyrecall
Forgetting detection and skill rollback for fine-tuned LLMs.
Fine-tune on new data and your model quietly loses what it already knew — coding ability, reasoning, safety guardrails. pyrecall catches it before it ships.
Install
pip install pyrecall # Python 3.11–3.14 · CUDA, MPS, and CPU
Quickstart
from pyrecall import Model
model = Model("Qwen/Qwen2.5-1.5b-Instruct")
model.snapshot("before")
model.learn("data.jsonl", epochs=3)
if not model.check().is_healthy:
model.rollback(to="before")
pyrecall init # default model used is Qwen/Qwen2.5-1.5B-Instruct
pyrecall snapshot before_v1 # baseline before training
pyrecall learn train.jsonl --snapshot-after after_v1 # train + snapshot in one step
pyrecall check # compares last two snapshots
# exit 0 → ship exit 2 → pyrecall rollback before_v1
To continuously monitor for new snapshots
pyrecall check --watch
How it works
Benchmarks 180 prompts across 9 skill categories (reasoning, coding, safety, math, multilingual, and more) using log-likelihood scoring. After training, check diffs the scores and flags any category that drops past your threshold (default 10%). Only the LoRA adapter is stored per snapshot — a few hundred MB, not the full model.
Any causal LM on HuggingFace Hub is supported — Llama, Mistral, Phi, Gemma, Qwen, Falcon, GPT-2/Neo/J, and more. LoRA targets are auto-detected.
Docs
Full CLI reference, Python API, experiment tracker integrations (W&B, MLflow, Neptune), custom benchmarks, per-category thresholds, and more:
Contributing
git clone https://github.com/Arths17/Pyrecall
pip install -e ".[dev]"
pytest
Open an issue before large changes. MIT — LICENSE.
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pyrecall-0.12.1.tar.gz.
File metadata
- Download URL: pyrecall-0.12.1.tar.gz
- Upload date:
- Size: 183.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.14.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6898678e8179b9b044f3348d9c84389465d685c2eaabda96ad19c3bac1c1c1a2
|
|
| MD5 |
97b1538a9d8effa85edd4a3765888d70
|
|
| BLAKE2b-256 |
2855bed475b8474102d23db04d60340fe2f33dd5c5a90f29cd34f17390e642b1
|
File details
Details for the file pyrecall-0.12.1-py3-none-any.whl.
File metadata
- Download URL: pyrecall-0.12.1-py3-none-any.whl
- Upload date:
- Size: 105.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.14.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
89e9f99158f40a03a0e16718fe54c5c949743e57bd1f49e24bc2b736ca4d2413
|
|
| MD5 |
46a086d22eb71fc775dd50b36761b616
|
|
| BLAKE2b-256 |
2b539cadae740f427d68e18a3540850ef021ab9c711af37dd7c939ec844fd1fd
|
