jetstream

Automated experiment analysis.

Jetstream automatically calculates metrics and applies statistical treatments to collected experiment data for different analysis windows.

For more information, see the documentation.

Deployment

Jetstream automatically builds and pushes a docker image when a PR is merged into the main branch.

Use the [ci breaking-change] prefix on a PR title in order to give the image the "breaking" tag, which helps Jetstream's automated versioning determine when an incompatible change has landed, and bump up to at least the latest compatible version so that analysis can succeed.

Local installation

# Create and activate a python virtual environment.
python3 -m venv venv/
source venv/bin/activate
# --no-deps to workaround https://github.com/pypa/pip/issues/9644
pip install --no-deps -r requirements.txt
pip install -e .

Running tests

• Install tox (included in requirements.txt)

• Run tox from wherever you cloned this repository. (You don't need to install jetstream first.)

  • To run integration tests, run tox -e py310-integration.

Linting / Formatting

We use ruff for linting and formatting:

• lint: ruff check jetstream
• format: ruff format --check jetstream

ruff can also fix (some) issues:

• lint: ruff check jetstream --fix
• format: ruff format jetstream

We also use mypy:

• mypy -p jetstream

You can also run tox py310-format to run all the ruff and mypy checks.

Dependencies

Jetstream uses pip-tools to manage dependencies, along with a script that runs the pip-tools commands. There are two requirements files auto-generated by script/update_deps:

• requirements.in: Listing of dependencies and versions generated from the dependencies list in pyproject.toml.
• requirements.txt: Auto-generated by pip-tools (pip-compile) from the requirements.in file. Also contains the hashes of each package for verification by pip during installation, and comments showing lineage for each dependency.

Add a new dependency

1. Add to pyproject.toml list of dependencies.
2. Generate new requirements files with ./script/update_deps
3. Install with python -m pip install -r requirements.txt

Update all dependencies

./script/update_deps

Be sure to run pip install --no-deps -r requirements.txt and reinstall jetstream (pip install -e .) afterwards, and test functionality!

Update a single dependency

0. Prefer to update all dependencies (see above)!

1. Edit requirements.in

• mypy==1.8.0 --> mypy==1.9.0

2. Regenerate requirements.txt

• pip-compile --generate-hashes -o requirements.txt requirements.in
  • (Note: this is the last line of script/update_deps)

3. Install dependencies

• pip install --no-deps -r requirements.txt
• pip install -e .

4. Test!