calisim 1.0.0

A toolbox for the calibration and evaluation of simulation models.

---

PyPI | Documentation | API | Changelog | Examples | Releases | Docker | Binder

A toolbox for the calibration and evaluation of simulation models.

Table of contents

• Table of contents
• Introduction
• Features and Functionality
• Quickstart
• Installation
• Usage with Docker
• Usage with Binder
• Communication
• Contributions and Support
• License

Introduction

calisim is an open-source, low-code model calibration library that streamlines and standardises your workflows, while aiming to be as flexible and extensible as needed to support more complex use-cases. Using calisim will speed up your experiment cycle substantially and make you more productive.

calisim is primarily a wrapper around popular libraries and frameworks including Optuna, PyMC, scikit-learn, and emcee among many others. The design and simplicity of calisim was inspired by the scikit-learn and PyCaret libraries.

Features and Functionality

• A standardised and streamlined interface to multiple calibration procedures and libraries.
• A low-code library, allowing modellers to rapidly construct multiple workflows for many calibration procedures.
• An object-oriented programming architecture, allowing users to easily extend and modify calibration workflows for their own complex modelling use-cases.
• An unopinionated approach to working with simulation models, allowing users to calibrate both Python-based and non-Python-based models.
• Optional integration with PyTorch for access to more sophisticated Gaussian Process and deep learning surrogate models, state-of-the-art evolutionary algorithms, and deep generative modelling for simulation-based inference.

Quickstart

# Load imports
import numpy as np
import pandas as pd

from calisim.data_model import (
	DistributionModel,
	ParameterDataType,
	ParameterSpecification,
)
from calisim.example_models import LotkaVolterraModel
from calisim.optimisation import OptimisationMethod, OptimisationMethodModel
from calisim.statistics import MeanSquaredError
from calisim.utils import get_examples_outdir

# Get model
model = LotkaVolterraModel()
observed_data = model.get_observed_data()

# Specify model parameter distributions
parameter_spec = ParameterSpecification(
	parameters=[
		DistributionModel(
			name="alpha",
			distribution_name="uniform",
			distribution_args=[0.45, 0.55],
			data_type=ParameterDataType.CONTINUOUS,
		)
	]
)

# Define objective function
def objective(
	parameters: dict, simulation_id: str, observed_data: np.ndarray | None, t: pd.Series
) -> float | list[float]:
	simulation_parameters = dict(
		alpha=parameters["alpha"],
		beta=0.024, h0=34.0, l0=5.9,
		t=t, gamma=0.84, delta=0.026,
	)

	simulated_data = model.simulate(simulation_parameters).lynx.values
	metric = MeanSquaredError()
	discrepancy = metric.calculate(observed_data, simulated_data)
	return discrepancy

# Specify calibration parameter values
specification = OptimisationMethodModel(
	experiment_name="optuna_optimisation",
	parameter_spec=parameter_spec,
	observed_data=observed_data.lynx.values,
	outdir=get_examples_outdir(),
	method="tpes",
	directions=["minimize"],
	n_iterations=100,
	method_kwargs=dict(n_startup_trials=50),
	calibration_func_kwargs=dict(t=observed_data.year),
)

# Choose calibration engine
calibrator = OptimisationMethod(
	calibration_func=objective, specification=specification, engine="optuna"
)

# Run the workflow
calibrator.specify().execute().analyze()

# View the results
result_artifacts = "\n".join(calibrator.get_artifacts())
print(f"View results: \n{result_artifacts}")
print(f"Parameter estimates: {calibrator.get_parameter_estimates()}")

Installation

The easiest way to install calisim is by using pip:

pip install calisim

calisim's default installation will not include all optional dependencies. You may be interested in one or more extras:

# Install PyTorch extras
pip install calisim[torch]

# Install Hydra extras
pip install calisim[hydra]

# Install TorchX extras
pip install calisim[torchx]

# Install multiple extras
pip install calisim[torch,hydra,torchx]

Usage with Docker

You may also want to execute calisim inside of a Docker container. You can do so by running the following:

# Change the image version as needed
export CALISIM_VERSION=latest

# Get docker-compose.yaml file
wget https://raw.githubusercontent.com/Plant-Food-Research-Open/calisim/refs/heads/main/docker-compose.yaml

# Pull the image
docker compose pull calisim

# Run an example
docker compose run --rm calisim python examples/optimisation/optuna_example.py

# Pull and run the image with PyTorch dependencies included
# docker compose pull calisim_torch
# docker compose run --rm calisim_torch python examples/optimisation/botorch_example.py

Usage with Binder

You may wish to work with calisim using MyBinder. You can do so by accessing the following link.

Communication

Please refer to the following links:

• GitHub Discussions for questions.
• GitHub Issues for bug reports and feature requests.

Contributions and Support

Contributions are more than welcome. For general guidelines on how to contribute to this project, take a look at CONTRIBUTING.md.

For our community code of conduct, please also view CODE_OF_CONDUCT.md.

License

calisim is published under the Apache License (see LICENSE).

View all third party licenses (see third_party)