swarmtorch 0.1.3

Professional PyTorch library for 120+ metaheuristic optimization algorithms (Swarm, Evolutionary, Physics, Hybrid).

SwarmTorch 🐝🔥

The ultimate metaheuristic library for PyTorch

---

SwarmTorch is a high-performance, academic-grade library that brings 120+ metaheuristic algorithms to the PyTorch ecosystem. It enables gradient-free neural network training and state-of-the-art hyperparameter optimization (HPO) with a single, unified API.

Installation • Key Features • Benchmarks • Usage • Citation

---

🚀 Key Features

• 120+ Algorithms: Categorized into Swarm Intelligence, Evolutionary, Physics-based, Human-based, Bio-inspired, and Hybrids.
• Gradient-Free Training: Optimize weights for non-differentiable or complex loss landscapes directly as a PyTorch Optimizer.
• Deep HPO Integration: Replace Random/Grid search with intelligent, nature-inspired exploration.
• Research Ready: Includes full benchmarking suites, raw experimental data, and publication-quality visualizations.
• Highly Optimized: Leverages PyTorch's tensor operations for swarm-level parallelism.

---

📈 Research & Benchmarking Results

We conducted a massive-scale rigorous evaluation of 118 algorithms to benchmark their performance across various deep learning tasks.

1. Model Training (Weight Optimization)

SwarmTorch enables the training of neural networks without gradients. Several metaheuristics exhibit convergence stability comparable to standard gradient-based methods.

Figure 1: Convergence history comparing Swarm Optimizers (PSO, HHO, etc.) against Adam and SGD.

Figure 2: The Top 25 most effective weight optimizers ranked by final loss.

2. High-Density Distribution Analysis

We analyzed the reliability of each category. Swarm and Hybrid categories demonstrated the highest stability and lowest variance across multiple trials.

Figure 3: Statistical distribution of final loss across categories. Lower loss indicates superior optimization.

3. Hyperparameter Optimization (HPO) Benchmarks

Our metaheuristic searchers are designed to replace Random Search with more intelligent exploration strategies. 94.9% of our algorithms outperformed Random Search.

Figure 4: Success rate of metaheuristic searchers vs. the Random Search baseline.

4. The "Generalist" Frontier

We identified "Generalist" algorithms that excel at both weight optimization and hyperparameter tuning.

Figure 5: Scatter plot mapping Training Efficiency vs. HPO Accuracy. Elite generalists occupy the top-right quadrant.

---

📦 Installation

Using pip:

pip install swarmtorch

Using uv (Recommended):

uv add swarmtorch

---

💻 Usage

Model Weight Optimization

import torch.nn as nn
from swarmtorch.swarm.model_training import PSO

model = nn.Sequential(nn.Linear(10, 64), nn.ReLU(), nn.Linear(64, 1))
optimizer = PSO(model.parameters(), swarm_size=30)

def closure():
    optimizer.zero_grad()
    loss = criterion(model(inputs), targets)
    return loss

optimizer.step(closure)

Hyperparameter Tuning

from swarmtorch.swarm.hyperparameter_tuning import PSOSearch

searcher = PSOSearch(
    model_fn=build_model,
    param_space={'lr': (0.001, 0.1), 'hidden_dim': [32, 64]},
    train_fn=train_fn,
    iterations=50
)
best_params = searcher.search()

---

🤝 Acknowledgments & References

This library was developed with reference to the pyMetaheuristic library. We are grateful for their contributions to the metaheuristic optimization community.

📝 Citation

@software{swarmtorch2026,
  author = {Halleluyah Darasimi Oludele},
  title = {SwarmTorch: A PyTorch Library for 120+ Metaheuristic Optimization Algorithms},
  year = {2026},
  url = {https://github.com/hallelx2/swarmtorch}
}