quietzero 0.1.0

Neural Architecture Search with Swarm Intelligence Algorithms

QuietZero

Neural Architecture Search with Swarm Intelligence Algorithms

QuietZero is a Python library for automatically discovering optimal neural network architectures using various swarm intelligence optimization algorithms. It is an upgraded version of the original DeepSwarm with additional swarm algorithms and enhanced features.

Features

• 9 Swarm Intelligence Algorithms:

  • Ant Colony Optimization (ACO)
  • Particle Swarm Optimization (PSO)
  • Artificial Bee Colony (ABC)
  • Grey Wolf Optimizer (GWO)
  • Firefly Algorithm (FA)
  • Bat Algorithm (BA)
  • Cuckoo Search (CS)
  • Whale Optimization Algorithm (WOA)
  • Moth Flame Optimizer (MFO)

• PyTorch Backend: Full support for PyTorch models and training

• Checkpoint/Resume: Save and restore search progress

• TensorBoard Integration: Optional logging for visualization

• Parallel Evaluation: Multi-GPU support for faster search

Installation

From PyPI (Recommended)

pip install quietzero

From Source

git clone https://github.com/TashinAhmed/QuietZero.git
cd QuietZero
pip install -e .

With Optional Dependencies

# With TensorBoard support
pip install quietzero[tensorboard]

# With development tools
pip install quietzero[dev]

Quick Start

from quietzero import DeepSwarm, PyTorchBackend, AntColony

# Define your settings
settings = {
    'ant_count': 10,
    'max_iterations': 10,
    'search_space': {
        'conv2d': {
            'filters': [16, 32, 64, 128],
            'kernel_size': [3, 5, 7],
            'activation': ['relu', 'tanh'],
        },
        'pool2d': {
            'type': ['max', 'avg'],
            'size': [2],
        },
        'dense': {
            'units': [64, 128, 256, 512],
            'activation': ['relu', 'tanh'],
        },
    },
    'training': {
        'epochs': 10,
        'batch_size': 32,
        'learning_rate': 0.001,
    },
}

# Create backend and optimizer
backend = PyTorchBackend()
optimizer = AntColony(settings=settings)

# Create DeepSwarm instance
deepswarm = DeepSwarm(
    backend=backend,
    optimizer=optimizer,
    settings=settings
)

# Find optimal architecture
best_topology, best_reward = deepswarm.find_architecture(your_dataset)

Supported Algorithms

Ant Colony Optimization (ACO)

Inspired by ant foraging behavior. Ants deposit pheromones on paths, and subsequent ants follow these pheromone trails to discover good solutions. Particularly effective for discrete search spaces.

Reference: Dorigo, M., & Stützle, T. (2004). Ant Colony Optimization. MIT Press.

from quietzero import AntColony

optimizer = AntColony(settings={
    'ant_count': 10,
    'max_iterations': 10,
    'rho': 0.1,  # Pheromone evaporation rate
    'alpha': 1.0,  # Pheromone influence
    'beta': 2.0,   # Heuristic influence
})

Particle Swarm Optimization (PSO)

Inspired by bird flocking and fish schooling. Particles move through the search space guided by their personal best and the global best position.

Reference: Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95, 4, 1942-1948.

from quietzero import ParticleSwarm

optimizer = ParticleSwarm(settings={
    'swarm_size': 10,
    'max_iterations': 10,
    'w': 0.7,   # Inertia weight
    'c1': 1.5,  # Cognitive parameter
    'c2': 1.5,  # Social parameter
})

Artificial Bee Colony (ABC)

Inspired by honey bee foraging behavior. The colony consists of employed bees, onlooker bees, and scout bees that collectively search for optimal solutions.

Reference: Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization. Technical Report-TR06, Erciyes University.

from quietzero import ArtificialBeeColony

optimizer = ArtificialBeeColony(settings={
    'colony_size': 10,
    'max_iterations': 10,
    'limit': 5,  # Abandonment limit
})

Grey Wolf Optimizer (GWO)

Inspired by grey wolf leadership hierarchy. The algorithm mimics the social hierarchy and hunting behavior of grey wolves with alpha, beta, delta, and omega wolves.

Reference: Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69, 46-61.

from quietzero import GreyWolfOptimizer

optimizer = GreyWolfOptimizer(settings={
    'pack_size': 10,
    'max_iterations': 10,
})

Whale Optimization Algorithm (WOA)

Inspired by humpback whale bubble-net hunting. Whales use a spiral bubble-net feeding strategy to trap prey.

Reference: Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm. Advances in Engineering Software, 95, 51-67.

from quietzero import WhaleOptimization

optimizer = WhaleOptimization(settings={
    'population_size': 10,
    'max_iterations': 10,
    'b': 1,  # Spiral shape constant
})

Cuckoo Search (CS)

Inspired by cuckoo brood parasitism combined with Lévy flights. Cuckoos lay their eggs in host nests, and the algorithm uses Lévy flights for exploration.

Reference: Yang, X. S., & Deb, S. (2009). Cuckoo search via Lévy flights. World Congress on Nature & Biologically Inspired Computing, 210-214.

from quietzero import CuckooSearch

optimizer = CuckooSearch(settings={
    'population_size': 10,
    'max_iterations': 10,
    'pa': 0.25,  # Abandonment probability
    'beta': 1.5,  # Lévy exponent
})

Firefly Algorithm (FA)

Inspired by firefly flashing behavior. Fireflies attract each other based on brightness, which corresponds to solution quality.

Reference: Yang, X. S. (2008). Nature-Inspired Metaheuristic Algorithms. Luniver Press.

from quietzero import FireflyAlgorithm

optimizer = FireflyAlgorithm(settings={
    'population_size': 10,
    'max_iterations': 10,
    'alpha': 0.2,  # Randomization
    'beta0': 1.0,  # Base attractiveness
    'gamma': 1.0,  # Light absorption
})

Bat Algorithm (BA)

Inspired by bat echolocation. Bats use frequency tuning and loudness control to navigate and hunt prey.

Reference: Yang, X. S. (2010). A new metaheuristic bat-inspired algorithm. Nature Inspired Cooperative Strategies for Optimization, 284, 65-74.

from quietzero import BatAlgorithm

optimizer = BatAlgorithm(settings={
    'population_size': 10,
    'max_iterations': 10,
    'f_min': 0,    # Min frequency
    'f_max': 2,    # Max frequency
    'alpha': 0.9,  # Loudness decay
    'gamma': 0.9,  # Pulse rate increase
})

Moth Flame Optimizer (MFO)

Inspired by moth transverse orientation. Moths navigate by maintaining a fixed angle relative to light sources.

Reference: Mirjalili, S. (2015). Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-Based Systems, 89, 228-249.

from quietzero import MothFlameOptimizer

optimizer = MothFlameOptimizer(settings={
    'population_size': 10,
    'max_iterations': 10,
    'b': 1,  # Spiral shape constant
})

Advanced Features

Checkpoint/Resume

Save and restore search progress:

# Save checkpoint
deepswarm.save_checkpoint('checkpoint.json')

# Resume from checkpoint
deepswarm.load_checkpoint('checkpoint.json')
best_topology, best_reward = deepswarm.find_architecture(dataset)

TensorBoard Logging

Enable TensorBoard logging:

deepswarm = DeepSwarm(
    backend=backend,
    optimizer=optimizer,
    settings=settings,
    log_dir='./logs',  # Enable TensorBoard
)

Then visualize:

tensorboard --logdir=./logs

Parallel Evaluation

Use multiple GPUs for faster search:

deepswarm = DeepSwarm(
    backend=backend,
    optimizer=optimizer,
    settings=settings,
    n_workers=4,  # Use 4 parallel workers
)

Examples

See the examples/ directory for complete examples:

• MNIST classification
• CIFAR-10 classification
• Fashion-MNIST classification

API Reference

Core Classes

DeepSwarm

Main entry point for neural architecture search.

from deepswarm import DeepSwarm

deepswarm = DeepSwarm(
    backend=backend,      # Required: Backend for model training
    optimizer=optimizer,  # Required: Swarm algorithm instance
    settings=settings,    # Required: Configuration dictionary
    log_dir=None,         # Optional: TensorBoard log directory
    n_workers=1,          # Optional: Number of parallel workers
)

# Methods
best_topology, best_reward = deepswarm.find_architecture(dataset)
deepswarm.save_checkpoint('checkpoint.json')
deepswarm.load_checkpoint('checkpoint.json')

BaseOptimizer

Abstract base class for all optimizers.

from deepswarm.base import BaseOptimizer

class MyOptimizer(BaseOptimizer):
    def generate_topology(self):
        # Return (agent_id, topology, state)
        return agent_id, topology, state
    
    def update(self, agent_id, reward, state=None):
        # Update optimizer state
        pass
    
    def is_finished(self):
        # Return True when search is complete
        return self.iteration >= self.max_iterations

Optimizer Classes

All optimizers share a common interface:

# Common methods for all optimizers
optimizer.generate_topology()  # Returns (agent_id, topology, state)
optimizer.update(agent_id, reward, state)  # Updates with evaluation result
optimizer.is_finished()  # Returns True when search complete
optimizer.get_state()  # Returns checkpoint dictionary
optimizer.set_state(state)  # Restores from checkpoint
optimizer.global_best_reward  # Best reward found
optimizer.global_best_topology  # Best topology found

AntColony

from deepswarm.aco import AntColony

optimizer = AntColony(settings={
    'ant_count': 10,        # Number of ants per iteration
    'max_iterations': 10,   # Maximum iterations
    'rho': 0.1,             # Pheromone evaporation rate (0-1)
    'alpha': 1.0,           # Pheromone influence
    'beta': 2.0,            # Heuristic influence
    'q0': 0.9,              # Exploitation vs exploration
    'tau0': 1.0,            # Initial pheromone
    'search_space': {...},  # Neural architecture search space
})

ParticleSwarm

from deepswarm.pso import ParticleSwarm

optimizer = ParticleSwarm(settings={
    'swarm_size': 10,       # Number of particles
    'max_iterations': 10,   # Maximum iterations
    'w': 0.7,               # Inertia weight
    'c1': 1.5,              # Cognitive coefficient
    'c2': 1.5,              # Social coefficient
    'v_max': 0.5,           # Maximum velocity
    'search_space': {...},  # Neural architecture search space
})

ArtificialBeeColony

from deepswarm.abc import ArtificialBeeColony

optimizer = ArtificialBeeColony(settings={
    'colony_size': 10,      # Number of bees (employed + onlooker)
    'max_iterations': 10,   # Maximum iterations
    'limit': 5,             # Abandonment limit for food sources
    'search_space': {...},  # Neural architecture search space
})

GreyWolfOptimizer

from deepswarm.gwo import GreyWolfOptimizer

optimizer = GreyWolfOptimizer(settings={
    'pack_size': 10,        # Wolf pack size
    'max_iterations': 10,   # Maximum iterations
    'search_space': {...},  # Neural architecture search space
})

WhaleOptimization

from deepswarm.woa import WhaleOptimization

optimizer = WhaleOptimization(settings={
    'population_size': 10,  # Population size
    'max_iterations': 10,   # Maximum iterations
    'b': 1,                 # Spiral shape constant
    'a_decay': 2.0,         # Linear decay coefficient for 'a'
    'search_space': {...},  # Neural architecture search space
})

CuckooSearch

from deepswarm.cs import CuckooSearch

optimizer = CuckooSearch(settings={
    'population_size': 10,  # Number of nests
    'max_iterations': 10,   # Maximum iterations
    'pa': 0.25,             # Abandonment probability (0-1)
    'beta': 1.5,            # Lévy flight exponent (0-2)
    'search_space': {...},  # Neural architecture search space
})

FireflyAlgorithm

from deepswarm.fa import FireflyAlgorithm

optimizer = FireflyAlgorithm(settings={
    'population_size': 10,  # Number of fireflies
    'max_iterations': 10,   # Maximum iterations
    'alpha': 0.2,           # Randomization parameter
    'beta0': 1.0,           # Base attractiveness
    'gamma': 1.0,           # Light absorption coefficient
    'search_space': {...},  # Neural architecture search space
})

BatAlgorithm

from deepswarm.ba import BatAlgorithm

optimizer = BatAlgorithm(settings={
    'population_size': 10,  # Number of bats
    'max_iterations': 10,   # Maximum iterations
    'f_min': 0,             # Minimum frequency
    'f_max': 2,             # Maximum frequency
    'alpha': 0.9,           # Loudness decay rate
    'gamma': 0.9,           # Pulse rate increase rate
    'search_space': {...},  # Neural architecture search space
})

MothFlameOptimizer

from deepswarm.mfo import MothFlameOptimizer

optimizer = MothFlameOptimizer(settings={
    'population_size': 10,  # Number of moths
    'max_iterations': 10,   # Maximum iterations
    'b': 1,                 # Spiral shape constant
    'search_space': {...},  # Neural architecture search space
})

Search Space Configuration

Define the neural architecture search space:

search_space = {
    'conv2d': {
        'filters': [16, 32, 64, 128],
        'kernel_size': [3, 5, 7],
        'activation': ['relu', 'tanh', 'sigmoid'],
        'padding': ['same', 'valid'],
    },
    'pool2d': {
        'type': ['max', 'avg'],
        'size': [2],
    },
    'dense': {
        'units': [64, 128, 256, 512],
        'activation': ['relu', 'tanh', 'sigmoid'],
    },
    'dropout': {
        'rate': [0.1, 0.2, 0.3, 0.5],
    },
    'training': {
        'optimizer': ['adam', 'sgd', 'rmsprop'],
        'learning_rate': [0.001, 0.0001, 0.00001],
        'batch_size': [16, 32, 64],
    }
}

Requirements

• Python >= 3.7
• PyTorch >= 1.7.0
• NumPy >= 1.19.0
• PyYAML >= 5.0

License

MIT License - see LICENSE for details.

Citation

If you use QuietZero in your research, please cite:

@misc{quietzero2024,
  author = {Tashin Ahmed},
  title = {QuietZero: Neural Architecture Search with Swarm Intelligence},
  year = {2024},
  publisher = {GitHub},
  url = {https://github.com/TashinAhmed/QuietZero}
}

Original DeepSwarm

This project is based on DeepSwarm by Pattio.

@misc{deepswarm2019,
  author = {Pattio},
  title = {DeepSwarm: Neural Architecture Search with Ant Colony Optimization},
  year = {2019},
  publisher = {GitHub},
  url = {https://github.com/Pattio/DeepSwarm}
}

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.