bees_algorithm 2.0.0

A Python implementation of the Bees Algorithm. This library allows an out-of-the-box use of the optimisation algorithm on an user-defined target function. The algorithm can be configured to find either the minimum or the maximum of the target function with an iterative process.

BeesAlgorithm - A Python Implementation

This repository contains a Bees Algorithm implementation in Python 3. For a more complete documentation please refer to the project page.

The aim is to make available to everyone an implementation, built with the minimal number of dependencies, which can be easily integrated in larger projects as well as used out-of-the-box to solve specific problems.

The Bees Algorithm is an intelligent optimization technique belonging to the swarm algorithms field. Given a parametric objective function, the goal of the algorithm is to find the parameter values that maximise/minimise the output of the objective function.

Many real-world problems can be modeled as the optimisation of a parametric objective function, therefore effective algorithms to handle this kind of problems are of primary importance in many fields. The Bees Algorithm performs simultaneus aggressive local searches around the most promising parameter settings of the objective function. The algorithm is proven to outperform other intelligent optimisation techniques in many benchmark functions^[3][4][5] as well as real world problems.

On top of this Python version, implmentations of the bees Algorithm in C++ and Matlab are also available in the respective repositories.

The main steps of the Bees Algorithm will be described in the next section. For more information please refer to the official Bees Algorithm website and the wikipedia page. If you are interested in a detailed analysis of the algorithm, and the properties of its search strategy, please refer to this paper^[1]:

• Luca Baronti, Marco Castellani, and Duc Truong Pham. "An Analysis of the Search Mechanisms of the Bees Algorithm." Swarm and Evolutionary Computation 59 (2020): 100746.

If you are using this implementation of the Bees Algorithm for your research, feel free to cite this work in your paper using the following BibTex entry:

@article{baronti2020analysis,
  title={An Analysis of the Search Mechanisms of the Bees Algorithm},
  author={Baronti, Luca and Castellani, Marco and Pham, Duc Truong},
  journal={Swarm and Evolutionary Computation},
  volume={59},
  pages={100746},
  year={2020},
  publisher={Elsevier},
  doi={10.1016/j.swevo.2020.100746},
  url={https://doi.org/10.1016/j.swevo.2020.100746}
}

Installation

This module requires Python 3.10 or newer and is available on pip:

$ pip install bees_algorithm

or, using uv:

$ uv add bees_algorithm

The runtime dependencies are numpy and pydantic. The step-by-step visualisation (see the relevant section below) also needs matplotlib, which is available through the optional plot extra:

$ pip install "bees_algorithm[plot]"

Introduction on the Bees Algorithm

The Bees Algorithm is a nature-inspired search method that mimics the foraging behaviour of honey bees. It was created by Prof. D.T. Pham and his co-workers in 2005^[2], and described in its standard formulation by Pham and Castellani^[3].

The algorithm uses a population of agents (artificial bees) to sample the solution space. A fraction of the population (scout bees) searches randomly for regions of high fitness (global search). The most successful scouts recruit a variable number of idle agents (forager bees) to search in the proximity of the fittest solutions (local search). Cycles of global and local search are repeated until an acceptable solution is discovered, or a given number of iterations have elapsed.

The standard version of the Bees Algorithm includes two heuristics: neighbourhood shrinking and site abandonment. Using neighbourhood shrinking the size of the local search is progressively reduced when local search stops progressing on a given site. The site abandonment procedure interrupts the search at one site after a given number of consecutive stagnation cycles, and restarts the local search at a randomly picked site.

The algorithm requires a number of parameters to be set, namely: the number of scout bees (ns), number of sites selected out of ns visited sites (nb), number of elite sites out of nb selected sites (ne), number of bees recruited for the best ne sites (nre), number of bees recruited for the other (nb-ne) selected sites (nrb). The heuristics also require the set of the initial size of the patches (ngh) and the number of cycles after which a site is abandoned (stlim). Finally, the stopping criterion must be defined.

The algorithm starts with the ns scout bees being placed randomly in the search space and the main algorithm steps can be summarised as follows:

1. Evaluate the fitness of the population according the objective function;
2. Select the best nb sites for neighbourhood (local) search;
3. Recruit nrb forager bees for the selected sites (nre bees for the best ne sites) and evaluate their fitnesses;
4. Select the fittest bee from each local site as the new site centre;
5. If a site fails to improve in a single local search, its neighbourhood size is reduced (neighbourhood shrinking);
6. If a site fails to improve for stlim cycles, the site is abandoned (site abandonment);
7. Assign the remaining bees to search uniformly the whole search space and evaluate their fitnesses;
8. If the stopping criterion is not met, return to step 2;

Usage

There is one implementation of the algorithm, BeesAlgorithm, and the way it is run is a parameter rather than a class of its own: hand it any concurrent.futures.Executor and the local searches of an iteration are spread over it. Two things sit beside it, because they are genuinely different and not just differently executed:

• AsyncBeesAlgorithm, a variant of the algorithm in which every site is searched by an independent process and the workers barely synchronise. It gives up the elite sites and the global search to do so, and it is not the standard algorithm any more;
• run_trials, which repeats independent optimisations of the same problem to assess a choice of parameters. That is a benchmarking helper, not a version of the algorithm.

Guidelines

• A single search, on an objective function that is cheap to evaluate: BeesAlgorithm, as it comes;
• A single search on an expensive objective function: BeesAlgorithm with an executor. The gain comes from the objective being slow enough to pay for handing the work to another process; on a cheap function the dispatch costs more than it saves;
• An objective function that can score many points at once (numpy, torch, a simulator with a batch API): BeesAlgorithm(..., vectorized=True), which is usually worth more than any amount of parallelism;
• A single search on a very expensive objective, where losing the elite sites is an acceptable price for near-perfect scaling: AsyncBeesAlgorithm;
• Assessing a set of parameters over many runs: run_trials.

A first search

Import the algorithm and describe the problem:

from bees_algorithm import BeesAlgorithm, SearchSpace

def hypersphere(x):
    return sum(xi ** 2 for xi in x)

space = SearchSpace(lower=[-5, -5, -5, -5], upper=[5, 5, 5, 5])

SearchSpace is a pydantic model, so a search space that makes no sense is rejected as it is built, not several thousand evaluations later:

>>> SearchSpace(lower=[0, 0], upper=[1])
ValidationError: the sizes of the lower and upper bounds don't match (2 != 1)

Then create the algorithm and run it:

algorithm = BeesAlgorithm(hypersphere, space, direction="minimize")
result = algorithm.optimize(target_score=0.001, max_iterations=5000)

result.score       # 0.0009539..., in the units of your objective function
result.position    # array([-0.00808, 0.01606, 0.01879, 0.00792])
result.iterations  # 27
result.stop_reason # 'target_score'

direction decides whether the objective is maximised or minimised: there is no need to write the negation of a function to minimise it, and target_score is always expressed in the units of the objective.

For a reproducible run, pass a seed. Every run of this library is fully determined by it, including the parallel ones:

BeesAlgorithm(hypersphere, space, direction="minimize", seed=42)

Setting the parameters

The parameters of the search are a model of their own, BeesParameters. Given none, the algorithm uses the customary values below. Note that n_scouts counts only the bees kept for the global search, so these defaults perform slightly more global search than the same figures read in the traditional formulation would (see below):

from bees_algorithm import BeesParameters

parameters = BeesParameters(
    n_scouts=10,          # ns:    scouts assigned to the global search
    n_sites=5,            # nb:    best sites kept at each iteration
    n_elite_sites=1,      # ne:    of which promoted to elite
    n_foragers_site=10,   # nrb:   foragers recruited on a best site
    n_foragers_elite=15,  # nre:   foragers recruited on an elite site
    stagnation_limit=10,  # stlim: failed local searches before a site is abandoned
    shrink_factor=0.2,    #        fraction the neighbourhood shrinks by
)
algorithm = BeesAlgorithm(hypersphere, space, parameters, direction="minimize")

The names map to the ones used in the literature as follows:

paper	BeesParameters	meaning
ns	n_scouts	scout bees performing the global search
nb	n_sites	sites selected for local search
ne	n_elite_sites	elite sites among them
nrb	n_foragers_site	foragers recruited on a selected site
nre	n_foragers_elite	foragers recruited on an elite site
stlim	stagnation_limit	cycles before a site is abandoned
ngh	SearchSpace.initial_ngh	initial size of a patch, as a fraction of the search space

n_scouts is the number of scouts used exclusively for the global search, so n_scouts=0 means no global search at all. In the traditional formulation, ns is instead the whole scout population, of which nb become the best sites; build the parameters with from_total_population to use it, and the conversion is done for you:

BeesParameters.from_total_population(n_scouts=15, n_sites=5)  # n_scouts becomes 10

Invalid combinations are refused rather than silently repaired:

>>> BeesParameters(n_sites=3, n_elite_sites=4)
ValidationError: the number of elite sites is higher than the number of best sites (4 > 3)

The initial size of the local searches belongs to the search space, since it is measured against it, and defaults to the whole box:

SearchSpace(lower=[-5, -5], upper=[5, 5], initial_ngh=[0.1, 0.1])

Running the search

optimize runs to completion and needs at least one stop criterion, either as keywords or as a StopCriteria model; the first one met wins:

result = algorithm.optimize(max_iterations=5000, target_score=0.001, verbose=1)

An iteration at a time, to interleave the search with something else:

state = algorithm.step()
state.iteration  # 1
state.best       # the best solution found so far
state.sites      # the current best sites, in decreasing order of score

Or as an iterator, which is the easiest way to watch, log or stop a search on your own terms:

best_so_far, stalled = float("inf"), 0
for state in algorithm.iterate(max_iterations=1000):
    if state.best.score < best_so_far:
        best_so_far, stalled = state.best.score, 0
    else:
        stalled += 1
    if stalled == 50:  # a stop criterion of your own: 50 iterations without progress
        break

With no stop criterion iterate simply never ends, and it is up to you to break out of it.

At any point the state of the search is readable from the instance itself:

algorithm.best        # best solution so far, which may no longer be among the sites
algorithm.best.score
algorithm.best.position
algorithm.sites       # the current best sites
algorithm.iteration   # iterations performed so far

Every score handed back is in the units of your objective function, whichever direction is being optimised.

Objective functions that score many points at once

By default the objective is called once per candidate, with a one dimensional array of coordinates. If it can score a whole batch, declare it: the algorithm will then hand it a (n, d) array and expect n scores back.

import numpy as np

def hypersphere_batch(x):      # x is (n, d) rather than (d,)
    return np.sum(np.square(x), axis=1)

algorithm = BeesAlgorithm(hypersphere_batch, space, direction="minimize", vectorized=True)

This is the cheapest speed-up available, and on array-friendly objectives it is worth considerably more than running on several cores.

Running in parallel

On an executor

The local searches of an iteration are independent, so they can be handed to any executor. The algorithm is unchanged: the searches it performs, and the number of iterations it needs, are the same as when it runs on one core.

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor(max_workers=8) as executor:
    algorithm = BeesAlgorithm(hypersphere, space, parameters, executor=executor)
    result = algorithm.optimize(max_iterations=5000)

The executor belongs to you, so it can be reused across runs and it is yours to shut down. Anything implementing the Executor interface will do, ThreadPoolExecutor and third-party pools included.

Note that this pays off only when the objective function is expensive enough to cover the cost of sending the work to another process. On a function that takes microseconds the dispatch dominates and the search will be slower than on a single core.

The asynchronous variant

AsyncBeesAlgorithm searches every site in its own process from beginning to end, and the workers only synchronise when a site is abandoned. Almost all of the coordination overhead disappears, at a price:

• there are no elite sites, since ranking the sites means synchronising them;
• there is no global scout search, for the same reason;
• the number of iterations reported is approximate, and can be overestimated;
• a run cannot be stepped through, only run to completion. There is no step.

Every site is searched by a process of its own, so the number of sites cannot exceed the size of the pool; asking for more is reported with a warning rather than silently queueing searches that would only start once the run is over.

It owns the pool it runs on, so it is a context manager:

from bees_algorithm import AsyncBeesAlgorithm

parameters = BeesParameters(n_scouts=0, n_elite_sites=0, n_sites=8, n_foragers_site=10)

with AsyncBeesAlgorithm(hypersphere, space, parameters,
                        direction="minimize", n_processes=8) as algorithm:
    result = algorithm.optimize(target_score=0.001, max_iterations=5000)

The parameters it cannot honour are refused instead of being silently ignored:

>>> AsyncBeesAlgorithm(hypersphere, space, BeesParameters(n_elite_sites=1))
ValueError: AsyncBeesAlgorithm has no global search and no elite sites, so n_scouts=10 and
n_elite_sites=1 cannot be honoured. Set them to 0, or use BeesAlgorithm with an executor to
keep them.

Assessing a set of parameters

Being stochastic, a single run of the Bees Algorithm says very little about a choice of parameters. run_trials runs many independent optimisations in parallel and summarises them:

from bees_algorithm import run_trials

report = run_trials(hypersphere, space, parameters,
                    n_trials=100, max_iterations=5000, target_score=0.001,
                    direction="minimize", seed=42)

report.converged          # how many runs reached the target score
report.iterations         # the iterations each run took, as an array
report.scores             # the score each run reached
report.best               # the best solution over the whole batch
report.iteration_summary  # min, quartiles, max, mean and std dev of the iterations
report.score_summary

Each trial gets its own independent random stream derived from seed, so a seeded batch is reproducible no matter how the runs happen to be scheduled across processes. Pass n_processes to choose the degree of parallelism, or executor to run on a pool you already own.

Serialising a configuration or a result

Every model is a pydantic model, so a configuration can be written to disk, sent over the wire and read back, which makes an experiment reproducible from a file:

saved = parameters.model_dump_json()      # '{"n_scouts":10,"n_sites":5,...}'
BeesParameters.model_validate_json(saved)  # the same parameters, back again

space.model_dump_json()                    # '{"lower":[-5.0,...],"upper":[5.0,...],...}'
result.model_dump()

A note on multiprocessing

Both AsyncBeesAlgorithm and a ProcessPoolExecutor send the objective function to another process, so it has to be picklable: define it at the top level of a module rather than as a lambda or a closure. On the platforms whose default start method is spawn (Windows and macOS), the code creating the pool also has to sit behind the usual guard:

if __name__ == "__main__":
    report = run_trials(hypersphere, space, n_trials=100, max_iterations=5000)

Step-by-step Visualisation

On a two dimensional objective function the search can be watched as it happens. The plot is a consumer of iterate like any other, and lives in its own module since it needs matplotlib:

import benchmark_functions as bf
from bees_algorithm import BeesAlgorithm, BeesParameters, SearchSpace
from bees_algorithm.plotting import plot_iterations

function = bf.Schwefel(n_dimensions=2)
lower, upper = function.suggested_bounds()

algorithm = BeesAlgorithm(
    function,
    SearchSpace(lower=lower, upper=upper),
    BeesParameters(n_scouts=0, n_sites=14, n_elite_sites=1,
                   n_foragers_site=5, n_foragers_elite=30, stagnation_limit=10),
    direction="minimize",
)
plot_iterations(algorithm)

It waits for a key press between iterations; pass pause=0.5 to let it run on its own instead, and a stop criterion to end it.

Migrating from 1.x

Version 2.0 is a clean break: the parameter names, the method names and the two parallel classes have all changed. The correspondence is:

1.x	2.0
BeesAlgorithm(f, lower, upper, ns=10, nb=5, ...)	BeesAlgorithm(f, SearchSpace(lower=..., upper=...), BeesParameters(n_scouts=10, n_sites=5, ...))
ParallelBeesAlgorithm(..., n_processes=8)	BeesAlgorithm(..., executor=ProcessPoolExecutor(8))
FullyParallelBeesAlgorithm	AsyncBeesAlgorithm
BeesAlgorithmTester(...).run_tests(n_tests=50, ...)	run_trials(..., n_trials=50)
alg.performFullOptimisation(max_iteration=..., max_score=...)	alg.optimize(max_iterations=..., target_score=...)
alg.performSingleStep()	alg.step()
alg.best_solution.values / .score	alg.best.position / .score
alg.current_sites	alg.sites
alg.visualize_iteration_steps()	bees_algorithm.plotting.plot_iterations(alg)
useSimplifiedParameters=True (the default meaning of ns)	the default, BeesParameters(n_scouts=...)
useSimplifiedParameters=False	BeesParameters.from_total_population(n_scouts=..., n_sites=...)
implementing -g(x) to minimise g	direction="minimize"
tester.iterations5values	report.iteration_summary
(iterations, score) = alg.performFullOptimisation(...)	result.iterations, result.score

Two behaviours changed on top of the renaming:

• runs are now reproducible. The 1.x versions drew from the global numpy random state, which made a run impossible to repeat and, worse, gave the workers of a forked pool correlated random streams. Each instance now owns a generator seeded from seed, and every worker gets an independently spawned one;
• the results are objects rather than tuples, and every score is reported in the units of the objective function.

References

• [1]: Luca Baronti, Marco Castellani, and Duc Truong Pham. "An Analysis of the Search Mechanisms of the Bees Algorithm." Swarm and Evolutionary Computation 59 (2020): 100746.
• [2]: Pham, Duc Truong, et al. "The Bees Algorithm—A Novel Tool for Complex Optimisation Problems." Intelligent production machines and systems. 2006. 454-459.
• [3]: Pham, Duc Truong, and Marco Castellani. "The bees algorithm: modelling foraging behaviour to solve continuous optimization problems." Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 223.12 (2009): 2919-2938.
• [4]: Pham, Duc Truong, and Marco Castellani. "Benchmarking and comparison of nature-inspired population-based continuous optimisation algorithms." Soft Computing 18.5 (2014): 871-903.
• [5]: Pham, Duc Truong, and Marco Castellani. "A comparative study of the Bees Algorithm as a tool for function optimisation." Cogent Engineering 2.1 (2015): 1091540.

For more references please refer to the README file in the C++ repository.

Author and License

This library is developed and mantained by Luca Baronti (gmail address: lbaronti) and released under GPL v3 license.

Versions History

v2.0.0

A breaking release: the API has been reworked and the 1.x names are gone. See the Migrating from 1.x section of the README for the full correspondence.

• there is now a single implementation of the algorithm. ParallelBeesAlgorithm is gone: parallelism is a parameter, and passing any concurrent.futures.Executor to BeesAlgorithm spreads the local searches of an iteration over it, without changing the search it performs
• FullyParallelBeesAlgorithm is now AsyncBeesAlgorithm. It remains a separate class because it is a different algorithm, but it no longer inherits an interface it can't honour: rather than raising NotImplementedError on a single step, it simply has none, and it refuses the parameters it can't use instead of silently ignoring them
• BeesAlgorithmTester has been replaced by the run_trials function, returning a TrialReport instead of leaving the results on the instance
• all the inputs are now pydantic models: SearchSpace, BeesParameters and StopCriteria. A malformed configuration is rejected as it is built, with a precise error, and a configuration can be serialised to JSON and read back
• searches are now reproducible. The 1.x versions drew from the global numpy random state, which made a run impossible to repeat and gave the workers of a forked pool correlated random streams. Every instance now owns a generator seeded from seed, and each worker gets an independently spawned one
• added direction="minimize", so that minimising a function no longer requires implementing its negation. Every score is reported in the units of the objective function
• added vectorized=True, for objective functions that can score a whole batch of points at once
• added iterate(), yielding the state of the colony after each iteration, which replaces the keep_bees_trace flag
• the sampling of the bees is now vectorised, which makes the search substantially faster on cheap objective functions
• fixed the asynchronous variant re-reading the stop criteria from the shared state on every iteration of every worker. Each of those reads was a round trip to the manager process, and removing them makes that variant roughly two orders of magnitude faster
• the asynchronous variant no longer submits more site searches than it has processes. A queued search could only start once another had met a stop criterion, which is to say once the run was already over, and then burned CPU until it next checked. Asking for more sites than processes now warns and searches as many sites as there are processes
• the visualisation moved out of the algorithm and into bees_algorithm.plotting.plot_iterations
• the results of a run are objects (OptimisationResult, Solution) rather than tuples, and report why the search stopped
• the methods follow PEP 8: performFullOptimisation is now optimize, performSingleStep is step
• the parameters use explicit names (ns is n_scouts, nb is n_sites, ...) and the useSimplifiedParameters flag, whose meaning was documented backwards, is replaced by the BeesParameters.from_total_population constructor
• added pydantic as a dependency, requires numpy 1.22 or newer
• the package ships type annotations (py.typed)
• the tests moved out of the repository root and into a tests/ directory, and run with pytest. They are split by the question they answer: tests/unit/ checks each part of the library in isolation, one module per component, in a couple of seconds; tests/optimisation/ checks that the algorithm still solves the calibrated benchmark problems as efficiently as it used to, over a hundred runs each. The expected iteration counts have been recalibrated on this implementation, and are now collected in tests/optimisation/conftest.py
• the tests are no longer shipped with the package: neither the wheel nor the sdist carries them, where the 1.x sdist installed tests.py and utests.py
• the benchmarks are now tests/benchmarks.py, and select which one to run from the command line instead of requiring an edit of a __main__ block

v1.0.3

• fixed FullyParallelBeesAlgorithm.performFullOptimisation raising ValueError: Pool is still running on every call
• fixed the unit tests of the parallel algorithms silently skipping every case, and recalibrated their expected iterations per algorithm
• moved to uv package manager
• created CD task to automatically push a new version upon tagging v* on main
• requires Python 3.10 or newer

v1.0.2

• minor fixes
• minor code refactoring
• added numpy dependnency
• moved static method visualize_steps to instance method visualize_iteration_steps

v1.0.1

• minor fixes in the README

v1.0.0

• full stable release
• added unit tests
• made compliant with benchmark_functions v1.1.3

v0.1.1

• initial beta release