Skip to main content
https://badge.fury.io/py/vivarium-cluster-tools.svg Documentation Status

Vivarium cluster tools is a python package that makes running vivarium simulations at scale on a Slurm cluster easy.

Supported Python versions: 3.10, 3.11, 3.12, 3.13

You can install vivarium-cluster-tools from PyPI with pip:

pip install vivarium-cluster-tools

or build it from source by cloning the monorepo and installing this package:

git clone https://github.com/ihmeuw/vivarium-suite.git
cd vivarium-suite
pip install libs/cluster-tools

A simple example

If you have a vivarium model specification file defining a particular model, you can use that along side a branches file to launch a run of many simulations at once with variations in the input data, random seed, or with different parameter settings.

psimulate run /path/to/model_specification.yaml /path/to/branches_file.yaml

The simplest branches file defines a count of input data draws and random seeds to launch.

input_draw_count: 25
random_seed_count: 10

This branches file defines a set of simulations for all combinations of 25 input draws and 10 random seeds and so would run, in total, 250 simulations.

You can also define a set of parameter variations to run your model over. Say your original model specification looked something like

plugins:
  optional: ...

components:
  vivarium.public_health:
    population:
      - BasePopulation()
    disease.models:
      - SIS('lower_respiratory_infections')
  my_lri_intervention:
    components:
      - GiveKidsVaccines()

configuration:
  population:
    population_size: 1000
    age_start: 0
    age_end: 5
  lri_vaccine:
    coverage: 0.2
    efficacy: 0.8

Defining a simple model of lower respiratory infections and a vaccine intervention. You could then write a branches file that varied over both input data draws and random seeds, but also over different levels of coverage and efficacy for the vaccine. That file would look like

input_draw_count: 25
random_seed_count: 10

branches:
  - lri_vaccine:
      coverage: [0.0, 0.2, 0.4, 0.8, 1.0]
      efficacy: [0.4, 0.6, 0.8]

The branches file would overwrite your original lri_vaccine configuration with each combination of coverage and efficacy in the branches file and launch a simulation. More, it would run each coverage-efficacy pair in the branches for each combination of input draw and random seed to produce 25 * 10 * 5 * 3 = 3750 unique simulations.

Multi-step workflows with dagger

For pipelines that chain several steps together, dagger runs a multi-step Jobmon workflow defined in a YAML configuration file. Each step lists its command and compute resources:

workflow:
  name: my_pipeline
  project: proj_simscience
  queue: all.q
  output_directory: /path/to/output
  steps:
    - name: launch_sims
      command: psimulate run /path/to/model_specification.yaml /path/to/branches_file.yaml
      resources:
        memory_gb: 4
        runtime: "01:00:00"
    - name: post_process
      command: my_post_processing_script /path/to/output
      resources:
        memory_gb: 8
        runtime: "00:30:00"

Launch the workflow with

dagger run -c /path/to/workflow.yaml

If a run fails partway through, resume it from its output directory, skipping steps that already completed, with

dagger restart /path/to/output

To read about more of the available features and get a better understanding of how to correctly write your own branches files,

Check out the docs!

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

vivarium_cluster_tools-4.4.0.tar.gz (173.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

vivarium_cluster_tools-4.4.0-py3-none-any.whl (93.5 kB view details)

Uploaded Python 3

File details

Details for the file vivarium_cluster_tools-4.4.0.tar.gz.

File metadata

  • Download URL: vivarium_cluster_tools-4.4.0.tar.gz
  • Upload date:
  • Size: 173.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for vivarium_cluster_tools-4.4.0.tar.gz
Algorithm Hash digest
SHA256 5fae1556c4e106b3f771aa9c5848ad4236c47c91a0520d6e7a4cf2bcc672a3b1
MD5 c2efbcb017c1b5133d222e3d35817b03
BLAKE2b-256 9e6b74eb4510b8445f8fc45cee1c03bdd3fc0957d728c3c3c98a00cf39d73d55

See more details on using hashes here.

Provenance

The following attestation bundles were made for vivarium_cluster_tools-4.4.0.tar.gz:

Publisher: release.yml on ihmeuw/vivarium-suite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file vivarium_cluster_tools-4.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for vivarium_cluster_tools-4.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0f4681d7bb4f7889378e43d543d5e858b1c786d124aafc122e4a1c34aac4b449
MD5 cff2014b4bd7a087c12b9450584669ee
BLAKE2b-256 f01c4216d305af107e41bb3213c6ecf885abea9863b75e4e80b2522386804ee5

See more details on using hashes here.

Provenance

The following attestation bundles were made for vivarium_cluster_tools-4.4.0-py3-none-any.whl:

Publisher: release.yml on ihmeuw/vivarium-suite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

4.6.0

2 files

4.5.0

2 files

This release

4.4.0 This release

2 files

4.3.0

2 files

4.2.14

2 files

4.2.13

2 files

4.2.12

2 files

4.2.11

2 files

4.2.10

2 files

4.2.9

2 files

4.2.8

2 files

4.2.7

2 files

4.2.6

2 files

4.2.5

2 files

4.2.4

2 files

4.2.3

2 files

4.2.2

2 files

4.2.1

2 files

4.2.0

2 files

4.1.0

2 files

4.0.1

2 files

4.0.0

2 files

3.1.6

2 files

3.1.5

2 files

3.1.4

2 files

3.1.3

2 files

3.1.2

2 files

3.1.1

2 files

3.1.0

2 files

3.0.0

2 files

2.3.5

2 files

2.3.4

2 files

2.3.3

2 files

2.3.2

2 files

2.3.1

2 files

2.3.0

2 files

2.2.2

2 files

2.2.1

2 files

2.2.0

2 files

2.1.26

2 files

2.1.25

2 files

2.1.24

2 files

2.1.23

2 files

2.1.22

2 files

2.1.21

2 files

2.1.20

2 files

2.1.19

2 files

2.1.18

2 files

2.1.17

2 files

2.1.16

2 files

2.1.15

2 files

2.1.14

2 files

2.1.13

2 files

2.1.12

2 files

2.1.11

2 files

2.1.10

2 files

2.1.9

2 files

2.1.8

2 files

2.1.7

2 files

2.1.6

2 files

2.1.5

2 files

2.1.4

2 files

2.1.3

2 files

2.1.2

2 files

2.1.1

2 files

2.1.0

2 files

2.0.4

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.6.1

2 files

1.6.0

2 files

1.5.6

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.13

2 files

1.3.12

2 files

1.3.11

2 files

1.3.10

2 files

1.3.9

2 files

1.3.8

2 files

1.3.7

2 files

1.3.6

2 files

1.3.5

2 files

1.3.4

2 files

1.3.3

2 files

1.3.1

2 files

1.3.0

2 files

1.2.13

2 files

1.2.12

2 files

1.2.11

2 files

1.2.10

2 files

1.2.9

2 files

1.2.8

2 files

1.2.6

2 files

1.2.5

2 files

1.2.4

2 files

1.2.3

2 files

1.2.1

2 files

1.2.0

2 files

1.1.2

1 file

1.1.1

1 file

1.1.0

1 file

1.0.15

1 file

1.0.14

1 file

1.0.12

1 file

1.0.11

1 file

1.0.10

1 file

1.0.9

1 file

1.0.8

1 file

1.0.7

1 file

1.0.6

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page