chai-sacred 0.8.3

Update pyptest version

If you update or change the pytest version, the following files need to be changed:

• dev-requirements.txt

• tox.ini

• test/test_utils.py

• setup.py

Omniboard

Omniboard is a web dashboard that helps in visualizing the experiments and metrics / logs collected by sacred. Omniboard is written with React, Node.js, Express and Bootstrap.

Incense

Incense is a Python library to retrieve runs stored in a MongoDB and interactively display metrics and artifacts in Jupyter notebooks.

Sacredboard

Sacredboard is a web-based dashboard interface to the sacred runs stored in a MongoDB.

Neptune

Neptune is a metadata store for MLOps, built for teams that run a lot of experiments. It gives you a single place to log, store, display, organize, compare, and query all your model-building metadata via API available for both Python and R programming languages:

In order to log your sacred experiments to Neptune, all you need to do is add an observer:

from neptune.new.integrations.sacred import NeptuneObserver
ex.observers.append(NeptuneObserver(api_token='<YOUR_API_TOKEN>',
                                    project='<YOUR_WORKSPACE/YOUR_PROJECT>'))

For more info, check the Neptune + Sacred integration guide.

SacredBrowser

SacredBrowser is a PyQt4 application to browse the MongoDB entries created by sacred experiments. Features include custom queries, sorting of the results, access to the stored source-code, and many more. No installation is required and it can connect to a local database or over the network.

Prophet

Prophet is an early prototype of a webinterface to the MongoDB entries created by sacred experiments, that is discontinued. It requires you to run RestHeart to access the database.

Sumatra

Sumatra is a tool for managing and tracking projects based on numerical

simulation and/or analysis, with the aim of supporting reproducible research.

It can be thought of as an automated electronic lab notebook for

computational projects.

Sumatra takes a different approach by providing commandline tools to initialize a project and then run arbitrary code (not just python). It tracks information about all runs in a SQL database and even provides a nice browser tool. It integrates less tightly with the code to be run, which makes it easily applicable to non-python experiments. But that also means it requires more setup for each experiment and configuration needs to be done using files. Use this project if you need to run non-python experiments, or are ok with the additional setup/configuration overhead.

Future Gadget Laboratory

FGLab is a machine learning dashboard, designed to make prototyping

experiments easier. Experiment details and results are sent to a database,

which allows analytics to be performed after their completion. The server

is FGLab, and the clients are FGMachines.

Similar to Sumatra, FGLab is an external tool that can keep track of runs from any program. Projects are configured via a JSON schema and the program needs to accept these configurations via command-line options. FGLab also takes the role of a basic scheduler by distributing runs over several machines.

CDE

By tracing system calls during program execution CDE creates a snapshot of all used files and libraries to guarantee the ability to reproduce any unix program execution. It only solves reproducibility, but it does so thoroughly.