finetuning-scheduler 0.1.1

A PyTorch Lightning extension that enhances model experimentation with flexible finetuning schedules.

A PyTorch Lightning extension that enhances model experimentation with flexible finetuning schedules.

---

Docs • Setup • Examples • Community

---

FinetuningScheduler is simple to use yet powerful, offering a number of features that facilitate model research and exploration:

• easy specification of flexible finetuning schedules with explicit or regex-based parameter selection
  • implicit schedules for initial/naive model exploration
  • explicit schedules for performance tuning, fine-grained behavioral experimentation and computational efficiency
• automatic restoration of best per-phase checkpoints driven by iterative application of early-stopping criteria to each finetuning phase
• composition of early-stopping and manually-set epoch-driven finetuning phase transitions

---

Setup

Step 0: Install from PyPI

pip install finetuning-scheduler

Step 1: Import the FinetuningScheduler callback and start finetuning!

from pytorch_lightning import Trainer
from finetuning_scheduler import FinetuningScheduler

trainer = Trainer(callbacks=[FinetuningScheduler()])

Get started by following the Finetuning Scheduler introduction which includes a CLI-based example or by following the notebook-based Finetuning Scheduler tutorial (link will be added as soon as it is released on the PyTorch Lightning production site).

---

Examples

Scheduled Finetuning For SuperGLUE

• Notebook-based Tutorial (link will be added as soon as it is released on the PyTorch Lightning production site)
• CLI-based Tutorial

---

Community

Finetuning Scheduler is developed and maintained by the community in close communication with the PyTorch Lightning team. Thanks to everyone in the community for their tireless effort building and improving the immensely useful core PyTorch Lightning project.

PR's welcome! Please see the contributing guidelines (which are essentially the same as PyTorch Lightning's).