sandevistanlr 0.2.0

ECG-inspired cyclical learning-rate scheduler with training modes

🚀 SandevistanLR — ECG-Inspired Cyclical Learning Rate

---

🧠 Overview

🚀 Abstraction
SandevistanLR is a step-level learning-rate scheduler in which the learning rate evolves as a deterministic, bounded time-domain signal.

⚡ Core Idea
Replace triangle or cosine schedules with a physiologically inspired ECG waveform (P–QRS–T) that explicitly encodes warm-up, exploration, and stabilization.

📈 Control Surface

• explicit phase boundaries
• bounded energy injection
• deterministic recovery

🔁 Execution Model

• evaluated per batch
• no optimizer state coupling
• no runtime heuristics

---

🫀 ECG Reference (P–QRS–T)

Public-domain reference:
SinusRhythmLabels.svg — Wikimedia Commons

---

📊 Learning-Rate Signal

🧩 Signal Semantics

• P wave → controlled warm-up
• QRS complex → short, high-energy exploration burst
• T wave → stabilization and consolidation

The waveform is:

• bounded
• repeatable
• phase-aware
• deterministic

---

⚙️ Optimization Modes

🎛 Explicit schedulable regimes

• normal — baseline ECG rhythm
• run — accelerated rhythm with higher exploration energy
• die — flatline (very low constant learning rate)
• aed — flatline with controlled shock spikes

Modes are:

• defined in advance
• deterministic
• never inferred from loss

---

🧪 Project Status & Scope

🚧 Status
This project is actively under development.

🎯 Goal
Produce a research-grade learning-rate scheduler supported by strong empirical evidence across vision and tabular benchmarks, with results packaged for conference-ready submission (e.g., CVPR / NeurIPS / ICLR workshops).

---

🎯 Motivation — LLMs & Short Training Runs

⚙️ Step-level execution
SandevistanLR operates per batch, not per epoch.

This is particularly relevant for large-scale training and fine-tuning, where runs are often limited to very few epochs (sometimes only one).

📉 In these regimes:

• epoch-level LR schedules become too coarse
• warm-up + decay collapses to near-constant behavior
• restart-based heuristics lack fine-grained phase control

🫀 ECG-style “heartbeat” cycles provide:

• controlled exploration
• bounded instability
• deterministic stabilization

even under extremely tight training budgets.

---

🧭 Preset Configurations (Recommended)

SandevistanLR ships with ready-to-use preset configurations covering a wide range of optimization behaviors:

• warmup → normal → cooldown
• athlete-like training cycles
• intermittent flatlines
• die → aed → recovery protocols
• interval training
• stress-test / arrhythmia-style schedules

These presets encode common optimization strategies in a reproducible and inspectable form.

---

🗂 Where to Find the Presets

📁 Preset definitions live in:

• src/sandevistanlr/configs.py

This file defines a dictionary of named configurations (CONFIGS) and their associated:

• schedule plans
• mode sequences
• timing parameters
• learning-rate bounds

Presets are strong starting points, not constraints.

SandevistanLR is fully parameterizable, allowing users to construct arbitrarily complex schedules by tuning:

• LR bounds
• cycle length
• ECG morphology
• mode transitions

to match their training budget and stability requirements.

---

🎞 Visual Previews

All preset waveforms are pre-generated and available in:

• figures/

Representative examples rendered below:

📊 Default ECG (normal)

---

📊 Warmup → Normal → Cooldown

---

📊 Die → AED → Recovery

---

📊 High-Energy Run Mode

These visualizations are intended to provide immediate, pre-training insight into each schedule’s behavior, enabling reasoning about optimization dynamics before integration into training pipelines.

---

📦 Installation

🚀 From PyPI

pip install sandevistanlr

🔧 From source

git clone https://github.com/takfarine/SandevistanLR.git

---

🔌 Usage

⚙️ PyTorch — Per-Step Control

from sandevistanlr import SandevistanScheduleBase
from sandevistanlr.adapters.pytorch import SandevistanLR

base = SandevistanScheduleBase(
    steps_per_epoch=100,
    epoch_plan=[("normal", 1), ("run", 6)],
    lr_min=1e-6,
    lr_max=2e-4,
)

sched = SandevistanLR(base)

for step in range(total_steps):
    lr = sched(step)
    for g in optimizer.param_groups:
        g["lr"] = lr

---

🔌 TensorFlow / Keras — Native Schedule

from sandevistanlr import SandevistanScheduleBase, SandevistanLRTF

base = SandevistanScheduleBase(
    steps_per_epoch=100,
    epoch_plan=[("normal", 1), ("run", 6)],
    lr_min=1e-6,
    lr_max=2e-4,
)

optimizer = tf.keras.optimizers.AdamW(
    learning_rate=SandevistanLRTF(base)
)

---

📐 Mathematical Formulation

📘 Full derivation is provided in:

• code_examples/sandevistanlr-mathematical-idea.ipynb

Separation of concerns:

• README → systems intuition & usage
• Notebook → equations & proofs

---

🧩 Design Principles

Core guarantees:

1. deterministic schedules
2. signal-based formulation
3. explicit phase control
4. cross-framework numerical parity

Suitable for:

• fine-tuning
• short-budget training
• controlled ablation studies
• reproducible benchmarks

---

🛣 Status & Roadmap

Current status: v0.1.0 — stable core

Planned:

• benchmark reports (vision, tabular)
• additional presets
• optional integration helpers

Not planned:

• heuristic auto-tuning
• randomized schedules
• opaque optimizer logic

---

📚 References

📈 Learning-Rate Scheduling

• Leslie N. Smith, Cyclical Learning Rates for Training Neural Networks (2015)
  https://arxiv.org/abs/1506.01186

• Leslie N. Smith & Nicholay Topin, Super-Convergence (2017)
  https://arxiv.org/abs/1708.07120

• Ilya Loshchilov & Frank Hutter, SGDR (2016)
  https://arxiv.org/abs/1608.03983

• fastai documentation, 1cycle policy
  https://docs.fast.ai/callback.schedule.html

🫀 ECG Foundations

• Willem Einthoven, Nobel Lecture (1925)
  https://www.nobelprize.org/prizes/medicine/1924/einthoven/lecture/

• Rivera-Ruiz et al., Einthoven’s String Galvanometer (2008)
  https://pmc.ncbi.nlm.nih.gov/articles/PMC2435435/

• Fisch, Centennial of the Electrocardiogram (2000)
  https://www.sciencedirect.com/science/article/pii/S0735109700009761

---

📜 License

Apache License 2.0
https://www.apache.org/licenses/LICENSE-2.0