sierra-research 1.5.8

Automation framework for the scientific method in R&D

Usage
Release
Development
Miscellaneous

TL;DR

What is SIERRA? See What is SIERRA?

Why should you use SIERRA? See Why SIERRA?

To install SIERRA (requires python 3.9+):

pip3 install sierra-research

SIERRA requires a recent OSX (tested with 13+) or Linux (tested with ubuntu 20.04+) and python >= 3.9. For more details, including the requirements for project code, see the SIERRA requirements.

To get started using SIERRA, see getting started.

Want to cite SIERRA? See Citing.

Have an issue using SIERRA? See Troubleshooting.

What is SIERRA?

SIERRA architecture, organized by pipeline stage, left to right. High-level inputs/outputs and active plugins and shown for each stage. “…” indicates areas of further extensibility and customization via new plugins. “Host machine” indicates the machine SIERRA was invoked on. The active plugins in each stage and what they cumulatively enable are highlighted in red.

SIERRA is a command line tool and plugin framework for:

• Automating R&D, providing faculties for seamless experiment generation, execution, and results processing.

• Accelerating R&D cycles by allowing researchers/developers to focus on the “science” aspects: developing new things and designing experiments to test them, rather than the engineering aspects (writing scripts, configuring environments, etc.).

• Improving the reproducibility of scientific research, particularly in AI.

Why SIERRA?

• It changes the paradigm of the engineering tasks researchers must perform from manual and procedural to declarative and automated. That is, from:

  "I need to perform these steps to run the experiment, process the data and
  generate the graphs I want."

  to:

  "Here is the environment and simulator/platforms(s) I want to use, the
  deliverables I want to generate, and the data I want to appear on them for
  my research query--GO!"

  Essentially, SIERRA handles the “engineering” parts of research on the backend, acting as a compiler of sorts, turning research queries into executable objects, running the “compiled” experiments, and processing results into visualizations or other deliverables.

• It has deep support for arbitrary parameter sweeps: numeric, categorical, or any combination thereof.

• It supports a wide range of execution engines/environments, and experiment input/output formats via plugins. SIERRA supports mix-and-match between all plugin types, subject to restrictions within the plugins themselves. This is and makes it very easy to run experiments on different hardware, targeting different simulators, in different execution environments, generating different outputs, etc., all with little to no configuration changes by the user.

• SIERRA maximizes reusability of code and configuration; it is designed so that no copy-pasting is ever needed, improving code quality with no additional effort from users.

• SIERRA has a rich model framework allowing you to run arbitrary models, generate data, and plot it on the same figure as empirical results–automatically.

• Why use SIERRA over something like prefect, dagster, or airflow ? Briefly, because SIERRA provides a common pipeline which is tested and can accommodate most use cases; SIERRA is not as feature complete as these other frameworks, though. For most use cases (but not all), that delta doesn’t matter. In addition, with the other frameworks, you have to create your own pipelines from scratch.

Not sure if SIERRA makes sense for you? Check out some of the use cases for which SIERRA was designed. If aspects of any sound familiar, then there is a strong chance SIERRA could help you! See the SIERRA docs to get started.

Citing

If you use SIERRA and have found it helpful, please cite the following paper:

@inproceedings{Harwell2022a-SIERRA,
author = {Harwell, John and Lowmanstone, London and Gini, Maria},
title = {SIERRA: A Modular Framework for Research Automation},
year = {2022},
isbn = {9781450392136},
publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
booktitle = {Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems},
pages = {1905–1907}
}

You can also cite the specific version of SIERRA used with the DOI at the top of this page, to help facilitate reproducibility.

Troubleshooting

If you have problems using SIERRA, please open an issue or post in the Github forum and I’ll be happy to help you work through it.

Contributing

I welcome all types of contributions, no matter how large or how small, and if you have an idea, I’m happy to talk about it at any point :-). See the contributing guide for the general procedure.