swarms-torch 0.1.0

swarms-torch - Pytorch

Swarms in Torch

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch primitives😊

Installation

You can install the package using pip

pip3 install swarms-torch

Usage

• We have just PSO now, but we're adding in ant colony and others!

from swarms_torch import ParticleSwarmOptimization

#test
pso = ParticleSwarmOptimization(goal="Attention is all you need", n_particles=100)
pso.optimize(iterations=1000)

• Ant Colony Optimization

from swarms_torch.ant_colony_swarm import AntColonyOptimization

# Usage:
goal_string = "Hello ACO"
aco = AntColonyOptimization(goal_string, num_iterations=1000)
best_solution = aco.optimize()
print("Best Matched String:", best_solution)

• Neural Network with Transformers as synapases LMAO.

import torch
from swarms_torch.nnt import NNTransformer

x = torch.randn(1, 10)

network = NNTransformer(
    #transformer cells
    neuron_count = 5, 
    
    #num states
    num_states = 10,

    #input dim
    input_dim = 10,

    #output dim
    output_dim = 10,

    #nhead
    nhead = 2,
)
output = network(x)
print(output)

• CellularSwarm, a Cellular Neural Net with transformers as cells, time simulation, and a local neighboorhood!

from swarms_torch import CellularSwarm 

x = torch.randn(10, 32, 512)  # sequence length of 10, batch size of 32, embedding size of 512
model = CellularSwarm(cell_count=5, input_dim=512, nhead=8)
output = model(x)

• Fish School/Sakana is an all-new innovative approaches to machine learning that leverage the power of the Transformer model architecture. These systems are designed to mimic the behavior of a school of fish, where each fish represents an individual Transformer model. The goal is to optimize the performance of the entire school by learning from the best-performing fish.

import torch
from swarms_torch.fish_school import Fish, FishSchool

# Create random source and target sequences
src = torch.randn(10, 32, 512)
tgt = torch.randn(10, 32, 512)

# Create random labels
labels = torch.randint(0, 512, (10, 32))

# Create a fish and train it on the random data
fish = Fish(512, 8, 6)
fish.train(src, tgt, labels)
print(fish.food)  # Print the fish's food

# Create a fish school and optimize it on the random data
school = FishSchool(10, 512, 8, 6, 100)
school.forward(src, tgt, labels)
print(school.fish[0].food)  # Print the first fish's food

Documentation

• Click here for documentation

Playground

• There are various scripts in the playground folder with various examples for each swarm, like ant colony and fish school and spiral optimization.

Todo

10 swarming neural network algorithms, with brief overviews, algorithmic pseudocode, and potential use cases for each:

1. Particle Swarm Optimization (PSO)

   • Overview: Simulates the social behavior of birds flocking or fish schooling. It adjusts trajectories of individual agents (particles) based on their own and their neighbors' best known positions.
   • Pseudocode:

     Initialize each particle with random position and velocity
     while not converged:
         for each particle:
             calculate fitness value
             if fitness better than its best, update its best
         end for
         for each particle:
             update velocity towards its best and global best
             update position
         end for
     end while
     

   • Use Cases: Function optimization, neural network training, feature selection.

2. Ant Colony Optimization (ACO)

   • Overview: Simulates the foraging behavior of ants to find paths through graphs. Uses pheromones to mark good paths, which evaporate over time.
   • Pseudocode:

     Initialize pheromones on paths
     while not converged:
         deploy ants to find paths based on pheromone strengths
         update pheromone strengths based on quality of paths
         evaporate some pheromone
     end while
     

   • Use Cases: Path finding, traveling salesman problem, network routing.

3. Bee Algorithm (BA)

   • Overview: Simulates the behavior of honey bees finding optimal nectar sources.
   • Pseudocode:

     Initialize scout bees randomly
     while not converged:
         deploy scout bees to search for nectar
         recruit forager bees based on nectar quality
         more foragers for better nectar sources
     end while
     

   • Use Cases: Job scheduling, function optimization, image processing.

4. Firefly Algorithm (FA)

   • Overview: Based on the flashing behavior of fireflies. Fireflies are attracted to each other depending on the brightness of their flashing.
   • Pseudocode:

     Initialize fireflies randomly
     while not converged:
         for each firefly:
             move towards brighter fireflies
         end for
     end while
     

   • Use Cases: Multi-modal optimization, feature selection, clustering.

5. Bat Algorithm (BA)

   • Overview: Inspired by the echolocation behavior of bats. Bats fly randomly and adjust positions based on emitted and returned echoes.
   • Pseudocode:

     Initialize bats with random positions and velocities
     while not converged:
         for each bat:
             adjust velocity based on echolocation feedback
             move bat
         end for
     end while
     

   • Use Cases: Engineering design, tuning machine learning algorithms, scheduling.

6. Wolf Search Algorithm (WSA)

   • Overview: Models the hunting behavior of gray wolves.
   • Pseudocode:

     Initialize wolves
     while not converged:
         calculate fitness of all wolves
         rank wolves: alpha, beta, delta, and omega
         adjust position of omega wolves towards alpha, beta, and delta
     end while
     

   • Use Cases: Neural network training, function optimization, game AI.

7. Fish School Search (FSS)

   • Overview: Simulates the social behavior of fish schooling.
   • Pseudocode:

     Initialize fish randomly in search space
     while not converged:
         for each fish:
             if neighbor has better food, move towards it
             else explore randomly
         end for
         adjust school behavior based on total food
     end while
     

   • Use Cases: Load balancing, function optimization, robotics.

8. Cuckoo Search (CS)

   • Overview: Based on the reproduction strategy of cuckoos. They lay eggs in host bird nests and those nests with the best eggs (solutions) will carry on to the next generation.
   • Pseudocode:

     Initialize host nests with eggs (solutions)
     while not converged:
         lay new eggs by Levy flights
         if new egg better than the worst in nest, replace it
         discover a fraction of nests and lay new eggs
     end while
     

   • Use Cases: Engineering design optimization, image processing, numerical optimization.

9. Whale Optimization Algorithm (WOA)

   • Overview: Simulates the bubble-net hunting strategy of humpback whales.
   • Pseudocode:

     Initialize whales
     while not converged:
         for each whale:
             encircle prey or search using bubble-net approach
         end for
     
     
     end while
     

   • Use Cases: Structural optimization, neural network training, function optimization.

10. Grasshopper Optimization Algorithm (GOA)

    • Overview: Simulates the swarming behavior of grasshoppers towards food sources.
    • Pseudocode:

      Initialize grasshoppers
      while not converged:
          for each grasshopper:
              adjust position towards other grasshoppers based on distance and food source
          end for
      end while
      

    • Use Cases: Job scheduling, clustering, neural network training.

License

MIT