topological-adam 1.0.3

Energy-Stabilized Topological Adam Optimizer for PyTorch

Energy-Stabilized Topological Adam Optimizer

A PyTorch optimizer implementing energy-stabilized α-β field coupling for gradient-based optimization. This approach combines adaptive moment estimation with topological field dynamics to improve performance on complex optimization landscapes.

Research Contribution

To our knowledge, this work represents the first implementation of energy-stabilized topological field dynamics in gradient-based optimization. The approach introduces several mathematical concepts not present in existing optimizers:

1. Topological Current Formulation: J = (α · ĝ) - (β · ĝ)
2. Coupled Field Evolution: Auxiliary fields evolve through differential equations
3. Energy Stabilization: Automatic field magnitude control via E = 0.5⟨α² + β²⟩
4. Topological Corrections: Parameter updates include tanh(α - β) terms

Empirical Evidence

Benchmark results on standard datasets demonstrate complexity-dependent improvements:

Dataset	Standard Adam	Topological Adam	Improvement
MNIST	97.70%	97.59%	-0.11%
Fashion-MNIST	87.97%	88.72%	+0.75%
CIFAR-10	68.31%	68.57%	+0.26%

This pattern suggests the algorithm provides benefits specifically where traditional optimizers face challenges - in complex optimization landscapes with multiple local minima.

Mathematical Foundation

Core Algorithm

The optimizer extends standard Adam with topological field dynamics:

# 1. Compute topological current
J = (α · ĝ) - (β · ĝ)  # where ĝ is normalized gradient

# 2. Evolve auxiliary fields
α ← (1-η)α + (η/μ₀)J·β
β ← (1-η)β - (η/μ₀)J·α

# 3. Apply energy stabilization
E = 0.5⟨α² + β²⟩
if E ≠ target_energy: rescale_fields(α, β)

# 4. Final parameter update
θ ← θ - lr[adam_direction + w_topo·tanh(α - β)]

Theoretical Background

The approach combines established principles from multiple domains:

• Adaptive moment estimation (proven Adam foundation)
• Field theory dynamics (coupled oscillator systems)
• Energy conservation principles (bounded field evolution)
• Topological methods (non-local gradient corrections)

The energy stabilization mechanism ensures numerical stability while the topological fields provide exploration capabilities beyond standard gradient descent methods.

Quick Start

import torch
from topological_adam import TopologicalAdam

# Replace Adam with Topological Adam
model = YourModel()
optimizer = TopologicalAdam(model.parameters(), lr=1e-3)

# Standard training loop
for batch in dataloader:
    optimizer.zero_grad()
    loss = compute_loss(model, batch)
    loss.backward()
    optimizer.step()
    
    # Monitor topological dynamics (optional)
    print(f"Energy: {optimizer.energy():.2e}, |J|: {optimizer.J_mean_abs():.2e}")

Installation

git clone https://github.com/yourusername/topological-adam.git
cd topological-adam
pip install torch torchvision matplotlib

Configuration

Default Usage

optimizer = TopologicalAdam(model.parameters(), lr=1e-3)

Advanced Configuration

optimizer = TopologicalAdam(
    model.parameters(),
    lr=1e-3,                    # Learning rate
    betas=(0.9, 0.999),        # Adam momentum coefficients
    eta=0.02,                   # Field evolution rate
    mu0=0.5,                   # Coupling strength
    w_topo=0.15,               # Topological correction weight
    target_energy=1e-3,        # Energy stabilization target
)

Parameter Guidelines

Parameter	Range	Effect	Recommendation
lr	1e-4 to 1e-2	Standard learning rate	Start with 1e-3
eta	0.01 to 0.05	Field evolution speed	0.02 for stability
w_topo	0.05 to 0.25	Topological influence	0.15 for balance
target_energy	1e-4 to 1e-2	Field magnitude control	1e-3 for most problems

Benchmark Results

Complete Performance Analysis

Run the full benchmark suite to reproduce results:

python benchmark.py

Expected output:

=== MNIST ===
Epoch 09 | Adam=97.70% | Topo=97.59% | Energy=5.762e-03 | |J|=2.669e-02

=== FASHION ===  
Epoch 09 | Adam=87.97% | Topo=88.72% | Energy=5.762e-03 | |J|=1.799e-02

=== CIFAR ===
Epoch 09 | Adam=68.31% | Topo=68.57% | Energy=7.683e-03 | |J|=3.383e-02

Stability Metrics

Energy Evolution:

• MNIST/Fashion-MNIST: 5.762e-03 (perfectly stable)
• CIFAR-10: 7.683e-03 (stable, adapted to problem complexity)

Topological Activity:

• Healthy range: 0.01-0.06 across all datasets
• No numerical instability observed during training

Performance Analysis

When Topological Adam Excels

Evidence suggests benefits in scenarios with:

✅ Complex optimization landscapes with multiple local minima
✅ Medium to high complexity problems (Fashion-MNIST, CIFAR-10)
✅ Non-convex loss surfaces requiring exploration
✅ Problems where standard optimizers plateau

When Standard Adam is Sufficient

⚪ Simple, convex-like problems (basic MNIST classification)
⚪ Memory-constrained environments (2× parameter overhead)
⚪ Applications where marginal improvements don't justify complexity

Computational Characteristics

• Time overhead: ~1.5× standard Adam per step
• Memory overhead: 2× parameter storage for auxiliary fields
• Benefit threshold: Improvements typically justify overhead on complex problems

Hardware Support

Automatic Device Detection

# Supports CPU, CUDA, and TPU automatically
optimizer = TopologicalAdam(model.parameters())
# Fields automatically placed on correct device

Platform Compatibility

• CPU: Standard CPU training
• CUDA: GPU acceleration with automatic device placement
• TPU: Google Cloud TPU with XLA compilation (torch-xla required)

Repository Structure

topological-adam/
├── topological_adam.py         # Main optimizer implementation
├── benchmark.py                # Complete benchmark suite
├── README.md                   # This documentation
├── requirements.txt            # Dependencies
└── .gitignore                  # Repository cleanup

Research Context

Related Work

While topology optimization and topological data analysis exist in machine learning, this represents the first optimizer to implement:

• Energy-stabilized field dynamics in gradient descent
• Topological current-driven parameter corrections
• Automatic energy control for numerical stability

Theoretical Implications

The approach suggests that incorporating physical principles (field dynamics, energy conservation) into optimization algorithms can provide measurable improvements on practical problems while maintaining computational tractability.

Contributing

We welcome contributions, particularly:

• Benchmark results on new datasets and architectures
• Theoretical analysis of convergence properties
• Parameter optimization strategies
• Memory efficiency improvements for large-scale models

Please see issues for current priorities.

Citation

If you use this optimizer in research, please cite:

@software{topological_adam_2025,
  title={Energy-Stabilized Topological Adam Optimizer},
  author={Steven Reid},
  year={2025},
  url={https://github.com/RRG314/topological-adam},
  note={Empirically validated improvements on Fashion-MNIST (+0.75\%) and CIFAR-10 (+0.26\%)}
}

License

This project is licensed under the MIT License - see LICENSE for details.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

Built on mathematical principles, validated through empirical evidence, designed for practical impact.