wcdfa

Weight-Change DFA — a real-time diagnostic for self-modifying systems.

Monitors the temporal structure of how a neural network modifies its own weights during training. Detects ordered-phase drift (sealing/rigidity), disordered-phase drift (dissolving/instability), and maintained criticality — using the DFA scaling exponent on the weight-update magnitude time series.

α > ~1.2  →  Ordered (sealing). System is consolidating rigidly.
α ≈  1.0  →  Critical. Simultaneously robust and flexible.
α < ~0.8  →  Disordered (dissolving). System cannot consolidate.

Unlike activity-based DFA, weight-change DFA measures self-modification dynamics directly and is not confounded by input statistics.

Install

pip install wcdfa                 # numpy only
pip install wcdfa[torch]          # + PyTorch integration
pip install wcdfa[all]            # + PyTorch + matplotlib

Quick start

from wcdfa import WeightChangeDFA

monitor = WeightChangeDFA(window=500)

for epoch in range(num_epochs):
    train_one_epoch(model, optimizer)
    monitor.update(model)

    if monitor.ready:
        print(f"Epoch {epoch}: α = {monitor.alpha:.3f} ({monitor.regime})")

Three lines in your training loop. That's it.

What it measures

At each training step, wcdfa computes ||ΔW|| — the Frobenius norm of the weight update across all parameter tensors. Over a rolling window, it applies Detrended Fluctuation Analysis (DFA) to this time series to extract the scaling exponent α.

The exponent tells you about the temporal structure of self-modification:

• α ≈ 1.0 — the weight updates have 1/f scaling (long-range correlations balanced with flexibility). The system is at criticality.
• α > 1.2 — the updates are too regular, too correlated. The system is sealing into the ordered phase. Consolidation is winning over flexibility. The basin is deepening without widening.
• α < 0.8 — the updates are too noisy, too uncorrelated. The system cannot consolidate. The basin is widening without deepening.

This is different from loss curves and gradient norms. A network can have a perfectly flat loss curve (it solved the task) while weight-change DFA shows α = 2.0 (it solved the task through rigid, sealed dynamics). The grokking literature confirms this: networks generalize ~12,000 epochs before their weight dynamics reach criticality.

API

WeightChangeDFA

monitor = WeightChangeDFA(
    window=500,              # samples before first DFA computation
    stride=100,              # recompute every N updates
    thresholds=(1.2, 0.8),   # (ordered, disordered) boundaries
)

RollingWeightChangeDFA

Extended version with epoch-level tracking, plotting, and logging:

from wcdfa import RollingWeightChangeDFA

monitor = RollingWeightChangeDFA(window=500)

for epoch in range(num_epochs):
    for batch in dataloader:
        train_step(model, batch, optimizer)
        monitor.update(model)
    monitor.end_epoch(epoch)

# After training
monitor.plot()                    # one-line visualization
monitor.plot(save_path="dfa.png") # save to file
epochs, alphas = monitor.epoch_history
print(monitor.summary())

Weights & Biases integration:

import wandb
wandb.init(project="my-training-run")
monitor = RollingWeightChangeDFA(window=500)

for epoch in range(num_epochs):
    train(model, optimizer)
    monitor.update(model)
    monitor.log_wandb(step=epoch)  # logs alpha + regime to wandb

compute_dfa

Standalone DFA computation for any 1D signal:

from wcdfa import compute_dfa

alpha, scales, fluctuations = compute_dfa(signal, min_box=4, max_box=None)

# With R² goodness-of-fit (how clean is the scaling?)
alpha, scales, fluctuations, r_sq = compute_dfa(signal, return_r_squared=True)
print(f"α = {alpha:.3f}, R² = {r_sq:.3f}")  # R² > 0.95 = clean scaling

Non-PyTorch usage

If you're using JAX, TensorFlow, or any other framework, compute ||ΔW|| yourself and pass it in:

monitor = WeightChangeDFA(window=500)

for step in range(num_steps):
    # Your training step here
    weight_norm = compute_your_weight_change_norm()
    monitor.update(weight_norm)

Interpreting results

The therapeutic window

The relationship between perturbation frequency and α traces a full phase transition:

Zero correction:    α ≈ 1.84 (deep ordered phase — sealed)
2% correction:      α ≈ 1.55 (27% of total effect from first 2%)
~40% correction:    α ≈ 1.02 (criticality)
95% correction:     α ≈ 0.80 (disordered phase — dissolved)

The first increment of corrective perturbation has an outsized effect. The most dangerous configuration for a self-modifying system is not insufficient correction but zero correction.

Four failure modes

Note: the captured return and return against self are not detectable by α alone — they require measuring the coupling between the system's attractor and its intended objective.

Key finding: 95.5% clean-step retention

When perturbation steps are stripped from the analysis, 95.5% of the DFA effect persists. The gap changes how the system modifies itself between perturbation events, not just during them.

Examples

See examples/ for:

• grokking_example.py — Reproducing the two-transition finding in modular addition
• basic_usage.py — Minimal PyTorch integration

Validation

The DFA implementation is validated against nolds, an established DFA package:

Over 50 white noise trials (N=1000): mean |Δ| = 0.018, max |Δ| = 0.057. Small differences are expected — nolds and wcdfa use slightly different scale selection strategies. Agreement within 0.05 is excellent for DFA.

Performance notes

Memory: The PyTorch integration clones all trainable parameters every step to compute ||ΔW||. For large models this adds memory overhead — roughly equal to the model's parameter memory on CPU. For models over ~1B parameters, compute ||ΔW|| directly from the optimizer state:

# Large-model approach: compute norm from optimizer state
monitor = WeightChangeDFA(window=500)

for step in range(num_steps):
    optimizer.zero_grad()
    loss.backward()
    
    # Compute ||ΔW|| from gradients × learning rate (approximation)
    total_sq = sum(
        (p.grad * lr).square().sum().item()
        for p in model.parameters() if p.grad is not None
    )
    norm = total_sq ** 0.5
    
    optimizer.step()
    monitor.update(norm)

Speed: DFA computation runs on a numpy array of length window (default 500). At 20 log-spaced scales, this takes <1ms — negligible compared to a training step. The bottleneck for large models is the parameter cloning, not the DFA.

Frozen parameters: Only parameters with requires_grad=True are tracked. Fine-tuning setups where most layers are frozen work correctly.

Gradient accumulation: If you use gradient accumulation, call monitor.update() after optimizer.step(), not after every loss.backward(). Between optimizer steps the weights don't change, producing zero norms that corrupt the DFA signal.

Background

Weight-change DFA was developed as part of a research program on constitutive gap dependence — the requirement that self-modifying systems periodically leave their operating regime and return to maintain dynamical criticality. The metric was introduced in:

Kogura, J. S. (2026). Does Your Model Need Sleep? Constitutive Gap Dependence and the Stability Problem in Self-Modifying AI.

The two-transition finding in grokking (generalization precedes criticality by ~12,000 epochs) was reported in:

Kogura, J. S. (2026). Grokking Precedes Criticality: Weight-Change DFA Reveals a Delayed Phase Transition in Generalizing Networks.

The theoretical framework is developed in:

Kogura, J. S. (2026). Constitutive Gap Dependence: A Temporal Mechanism for Criticality Maintenance in Self-Modifying Systems. Submitted to J. R. Soc. Interface.

Kogura, J. S. (2026). The Arriving Breath: A Philosophical Conspiracy — The Temporal Ground of Caring. ISBN 979-8-9954717-0-7.

More at caring-gap.com.

Citation

If you use wcdfa in your research, please cite:

@software{kogura2026wcdfa,
  author = {Kogura, Jimi Sadaki},
  title = {wcdfa: Weight-Change Detrended Fluctuation Analysis},
  year = {2026},
  url = {https://github.com/jimikogura/wcdfa},
}

@article{kogura2026sleep,
  author = {Kogura, Jimi Sadaki},
  title = {Does Your Model Need Sleep? Constitutive Gap Dependence and the Stability Problem in Self-Modifying AI},
  year = {2026},
  doi = {10.5281/zenodo.19389821},
}

@article{kogura2026grokking,
  author = {Kogura, Jimi Sadaki},
  title = {Grokking Precedes Criticality: Weight-Change DFA Reveals a Delayed Phase Transition in Generalizing Networks},
  year = {2026},
}

License

MIT. Use it, build on it, cite it.