sandevistanlr 0.1.0

ECG-inspired cyclical learning-rate scheduler with training modes

SandevistanLR — ECG-Inspired Cyclical Learning Rate

Official ECG template (P–QRS–T) — reference

Reference ECG diagram (Public Domain) from SinusRhythmLabels.svg (Wikimedia Commons).

SandevistanLR waveform (ECG-shaped learning rate)

A physiologically-inspired learning-rate scheduler where the LR waveform follows a realistic ECG template (P–QRS–T) instead of triangle/cosine curves.

Inspired by CLR, 1cycle, and warm restarts, SandevistanLR introduces four training modes:

• normal — standard ECG rhythm
• run — faster rhythm + exercise-like morphology changes
• die — flatline (very low constant LR)
• aed — flatline + controlled shock spikes (“defibrillation”)

Status: This project is actively under development.
The goal is to produce a research-grade scheduler with strong empirical evidence across vision + tabular benchmarks, with results packaged for a conference-ready submission (e.g., CVPR / NeurIPS / ICLR workshops).

Motivation (LLMs & short runs): SandevistanLR is step-based (per-batch updates), which makes it especially promising for large-scale training and fine-tuning where runs are often limited to very few epochs (sometimes only 1).
In these regimes, per-epoch LR schedules can be too coarse; step-level “heartbeat” cycles provide controlled exploration and stabilization even in short training budgets.

---

Preset Configurations (Recommended)

SandevistanLR ships with ready-to-use preset configurations that cover multiple training behaviors such as:

• warmup → normal → cooldown
• athlete-like training cycles
• intermittent flatlines
• die → aed → recovery protocols
• interval training
• stress-test / arrhythmia-style schedules

Where to find the presets

• Code (presets): src/sandevistanlr/configs.py
  This file defines a dictionary of named configs (CONFIGS) and their corresponding schedule plans and parameters. The preset configurations are intentionally provided as strong starting points.
  SandevistanLR is fully parameterizable, allowing users to generate infinitely many custom schedules by tuning LR bounds, cycle length, ECG morphology, and mode transitions to match their training budget and stability requirements.

Visual previews

All preset waveforms have been pre-generated and are available in figures/.

Below are a few previews directly rendered inside the README:

Default ECG (normal)

Warmup → Normal → Cooldown

Die → AED → Recovery

High-energy Run mode

These images are intended to provide an immediate visual understanding of each schedule’s behavior before integrating it into training.

---

Install

Option 1 — PyPI

pip install sandevistanlr

Option 2 — From source

git clone https://github.com/takfarine/SandevistanLR.git

---

Usage

Option 1 — PyTorch (per-step / per-batch)

import torch
from sandevistanlr import SandevistanLR

optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)

sched = SandevistanLR(
    lr_min=1e-6,
    lr_max=3e-4,
    steps_per_cycle=2000,
    mode="normal",  # normal | run | die | aed
)

global_step = 0
for epoch in range(EPOCHS):
    for batch in loader:
        # update LR per step
        lr = sched(global_step)
        for pg in optimizer.param_groups:
            pg["lr"] = lr

        loss = train_step(batch)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()

        global_step += 1

Option 2 — TensorFlow / Keras (manual LR assignment)

import tensorflow as tf
from sandevistanlr import SandevistanLR

optimizer = tf.keras.optimizers.AdamW(learning_rate=1e-3)

sched = SandevistanLR(
    lr_min=1e-6,
    lr_max=3e-4,
    steps_per_cycle=2000,
    mode="run",
)

global_step = tf.Variable(0, dtype=tf.int64)

@tf.function
def train_step(x, y):
    # update LR per step
    lr = sched(int(global_step.numpy()))
    optimizer.learning_rate.assign(lr)

    with tf.GradientTape() as tape:
        pred = model(x, training=True)
        loss = loss_fn(y, pred)

    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))

    global_step.assign_add(1)
    return loss

---

Code Examples

End-to-end examples are provided under code_examples/:

• PyTorch — code_examples/pytorch_demo.ipynb
• TensorFlow / Keras — code_examples/tensorflow_demo.ipynb
• Waveform plots — code_examples/plot_waveforms.ipynb

---

References

Learning-rate scheduling

• Leslie N. Smith, Cyclical Learning Rates for Training Neural Networks, 2015.
  https://arxiv.org/abs/1506.01186

• Leslie N. Smith and Nicholay Topin, Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates, 2017.
  https://arxiv.org/abs/1708.07120

• Ilya Loshchilov and Frank Hutter, SGDR: Stochastic Gradient Descent with Warm Restarts, 2016.
  https://arxiv.org/abs/1608.03983

• fastai documentation, The 1cycle policy.
  https://docs.fast.ai/callback.schedule.html

ECG foundations and original electrocardiography work

• Willem Einthoven, The String Galvanometer and the Measurement of the Action Currents of the Heart (Nobel Lecture, 1925).
  https://www.nobelprize.org/prizes/medicine/1924/einthoven/lecture/
  PDF: https://www.nobelprize.org/uploads/2018/06/einthoven-lecture.pdf

• M. Rivera-Ruiz et al., Einthoven’s String Galvanometer: The First Electrocardiograph, 2008.
  https://pmc.ncbi.nlm.nih.gov/articles/PMC2435435/

• C. Fisch, Centennial of the String Galvanometer and the Electrocardiogram, Journal of the American College of Cardiology, 2000.
  https://www.sciencedirect.com/science/article/pii/S0735109700009761

• Wikimedia Commons (Public Domain), SinusRhythmLabels.svg (ECG components diagram).
  https://commons.wikimedia.org/wiki/File:SinusRhythmLabels.svg

---

License

Apache License 2.0 — see LICENSE