Skip to main content

black-it: Black-box abm calibration kit

Project description

*

PyPI - Python Version GitHub DOI badge

Black-box abm calibration kit

Black-it is an easy-to-use toolbox designed to help you calibrate the parameters in your agent-based models and simulations (ABMs), using state-of-the-art techniques to sample the parameter search space, with no need to reinvent the wheel.

Models from economics, epidemiology, biology, logistics, and more can be dealt with. The software can be used as-is - if your main interest is the ABM model itself. However, in case your research thing is to, e.g., devise new sampling strategies for ginormous search spaces and highly non-linear model, then you can deploy and test your new ideas on a solid, reusable, modular foundation, in a matter of days, with no need to reimplement all the plumbings from scratch.

Installation

This project requires Python v3.8 or later.

To install the latest version of the package from PyPI:

pip install black-it

Or, directly from GitHub:

pip install git+https://github.com/bancaditalia/black-it.git#egg=black-it

If you'd like to contribute to the package, please read the CONTRIBUTING.md guide.

Installation on Apple Silicon machines

Due to a dependency on the tables package, installing Black-it on Apple Silicon machines can sometimes require specifying the path to the HDF5 executable as explained here.

Quick Example

The GitHub repo of Black-it contains a series ready-to-run calibration examples.

To experiment with them, simply clone the repo and enter the examples folder

git clone https://github.com/bancaditalia/black-it.git
cd black-it/examples

You'll find several scripts and notebooks. The following is the script named main.py, note that copying and pasting the lines below will not work in general as the script needs to be inside the "examples" folder in order to run correctly.

import models.simple_models as md

from black_it.calibrator import Calibrator
from black_it.loss_functions.msm import MethodOfMomentsLoss
from black_it.samplers.best_batch import BestBatchSampler
from black_it.samplers.halton import HaltonSampler
from black_it.samplers.random_forest import RandomForestSampler

true_params = [0.20, 0.20, 0.75]
bounds = [
    [0.10, 0.10, 0.10],  # LOWER bounds
    [1.00, 1.00, 1.00],  # UPPER bounds
]
bounds_step = [0.01, 0.01, 0.01]  # Step size in range between bounds

batch_size = 8
halton_sampler = HaltonSampler(batch_size=batch_size)
random_forest_sampler = RandomForestSampler(batch_size=batch_size)
best_batch_sampler = BestBatchSampler(batch_size=batch_size)

# define a model to be calibrated
model = md.MarkovC_KP

# generate a synthetic dataset to test the calibrator
N = 2000
seed = 1
real_data = model(true_params, N, seed)

# define a loss
loss = MethodOfMomentsLoss()

# define the calibration seed
calibration_seed = 1

# initialize a Calibrator object
cal = Calibrator(
    samplers=[halton_sampler, random_forest_sampler, best_batch_sampler],
    real_data=real_data,
    model=model,
    parameters_bounds=bounds,
    parameters_precision=bounds_step,
    ensemble_size=3,
    loss_function=loss,
    random_state=calibration_seed,
)

# calibrate the model
params, losses = cal.calibrate(n_batches=15)

print(f"True parameters:       {true_params}")
print(f"Best parameters found: {params[0]}")

When the calibration terminates (~half a minute), towards the end of the output you should see the following messages:

True parameters:       [0.2, 0.2, 0.75]
Best parameters found: [0.19 0.21 0.68]

Docs

Black-it calibration is initiated via the Calibrator which, when called, performs three main steps.

First, a Sampler is summoned to suggest a set of promising parameter configurations to explore.

Second, the model to be calibrated is simulated for all the selected parameters.

Third, a specific loss function, measuring the goodness of fitness of the simulation data with respect to the real data, is evaluated.

These steps are performed in a loop, and this allows the samplers to progress towards better parameter values by exploiting the knowledge of previously computed loss functions.

A more detailed explanation of how Black-it works is available here, while the full documentation -complete with examples and tutorials- is available here.

Citing Black-it

A description of the package is available here.

Please consider citing it if you found this package useful for your research

@article{black_it, 
  title = {Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models}, 
  journal = {Journal of Open Source Software},
  publisher = {The Open Journal}, 
  year = {2022}, 
  volume = {7}, 
  number = {79}, 
  pages = {4622}, 
  doi = {10.21105/joss.04622}, 
  url = {https://doi.org/10.21105/joss.04622}, 
  author = {Marco Benedetti and 
            Gennaro Catapano and 
            Francesco {De Sclavis} and 
            Marco Favorito and 
            Aldo Glielmo and 
            Davide Magnanimi and 
            Antonio Muci} 
}

Disclaimer

This package is an outcome of a research project. All errors are those of the authors. All views expressed are personal views, not those of Bank of Italy.


* Credits to Sara Corbo for the logo.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

black_it-0.3.1.tar.gz (61.9 kB view details)

Uploaded Source

Built Distribution

black_it-0.3.1-py3-none-any.whl (96.5 kB view details)

Uploaded Python 3

File details

Details for the file black_it-0.3.1.tar.gz.

File metadata

  • Download URL: black_it-0.3.1.tar.gz
  • Upload date:
  • Size: 61.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.9.19 Darwin/23.6.0

File hashes

Hashes for black_it-0.3.1.tar.gz
Algorithm Hash digest
SHA256 6bdeaab34fc1035bcad467c9b4b8451182e0220c65cc050a3f6d7f663eb71e4d
MD5 2468112352229d21a655ccab7a9dfb81
BLAKE2b-256 574c98817054753578f4c39c13b54819848e530f57c40614e5046261636552c0

See more details on using hashes here.

File details

Details for the file black_it-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: black_it-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 96.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.9.19 Darwin/23.6.0

File hashes

Hashes for black_it-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 2e3badf49ea91fe3219bc0df69acef9db1ff2e8ec3abb09c5e5f6b30778e73e5
MD5 754ced5d1662452ec61df6085484c4a2
BLAKE2b-256 1e48dea3749d0ac784697ebc5801c54f5f913e58c506294ae7a38fd1b7789c12

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page