bloomctl

Command-line tool for the Bloom Database (Salk Harnessing Plants Initiative) — log in, find cylinder experiments, download their images and metadata, and work with cylinder trait datasets.

Install

Releases are still pre-releases (0.1.0aN), so install by asking for the version by name:

uv tool install "bloomctl==0.1.0a7"    # isolated CLI tool (recommended)
uvx bloomctl@0.1.0a7 --help            # one-off, no install
pip install "bloomctl==0.1.0a7"        # into the active environment

bloomctl --version

Don't add --pre or --prerelease=allow. Those flags aren't specific to bloomctl — they let every dependency install an unfinished dev version too.

Quickstart for Cylinder Image Downloads

# 1. Log in once (prompts for your Bloom email + password; saves to ~/.bloom)
bloomctl login

# 2. Find an experiment — pick a species from a menu, grab its id
bloomctl cyl experiments list --species-menu

# 3. Download it — by id, or just by name
bloomctl cyl download ./out --experiment-id 42
bloomctl cyl download ./out --experiment-name "drought 2024"

That writes ./out/scans.csv (metadata) and the per-frame images.

Downloads run 8 frames at a time. On a fast connection you can raise that:

bloomctl cyl download ./out --experiment-id 42 --workers 16    # up to 64

If a download stops part-way, run the same command again. It keeps whatever is already on disk and fetches only what is missing, so nothing is downloaded twice.

Every command takes -p/--profile to target a different login (default prod), and the list commands take --output csv|json for machine-readable output.

Quickstart for Plate Image Downloads

Plate (GraviScan) experiments work the same way, with plate in place of cyl:

# 1. Log in once, if you have not already
bloomctl login

# 2. Download a whole plate experiment — by id, or just by name
bloomctl plate download ./gravi --experiment-id 12
bloomctl plate download ./gravi --experiment-name "gravitropism" --species Arabidopsis

That writes ./gravi/plates.csv and plate_sections.csv (metadata) and the plate images.

Narrow it down when you do not want the whole experiment:

bloomctl plate download ./gravi --experiment-id 12 --plate-id PLATE-001   # one plate
bloomctl plate download ./gravi --experiment-id 12 --wave-number 3        # one wave
bloomctl plate download ./gravi --experiment-id 12 --meta-only            # csv only, no images

Start with --meta-only. A continuous plate session captures one image per plate per cycle, so a multi-day experiment is thousands of files — the metadata tells you what you are about to pull. --workers raises the download concurrency here too, and --limit fetches at most that many scans, for looking at a sample — it is not a way to export an experiment in parts, so give a sample and a full download separate directories.

Resume works as it does for cylinder downloads, and a little better: plate images record their size, so a file is skipped only when its size matches the database. A download truncated by a dropped connection is re-fetched rather than treated as complete.

plate download needs the gravi_scans_extended view on the server, and gravi_experiment_search if you select by --experiment-name.

Quickstart for Single-cell Dataset Files

A single-cell dataset's whole AnnData file (.h5ad) is stored gzipped and named by the SHA-256 of the uncompressed file, so the same file uploaded twice is one object.

pip install 'bloomctl[scrna]'                  # upload's structure check needs h5py

bloomctl scrna hdf5 upload my_dataset.h5ad     # needs write access
bloomctl scrna hdf5 download "My dataset"      # by name, id, or --checksum
bloomctl scrna hdf5 list                       # what storage holds
bloomctl scrna hdf5 list --file my_dataset.h5ad  # is this file already stored?

upload checks the file's structure before sending anything, and an interrupted upload resumes when you run the same command again. download writes the file only once its fingerprint matches.

Commands

Find & download (any logged-in user):

Pipeline (stage-in / write-back):

Single-cell (pip install 'bloomctl[scrna]' for upload):

Run bloomctl <command> --help for the full options of any command.

The download log

Every cyl download writes download_log.txt next to scans.csv, with one line per frame and a summary at the bottom. It is the file to send us if something looks wrong.

OK   scan=1 frame=0 cyl-images/0.png
SKIP scan=1 frame=1 cyl-images/1.png
FAIL scan=1 frame=3 cyl-images/3.png  error=[Errno 28] No space left on device
UNLISTED scan=5 (frame count unknown)  error=...
NOFRAMES scan=7 (no images recorded for this scan)

Summary: 3/8 frames present (3 downloaded this run, 0 already on disk), 5 failed

A log full of SKIP is normal: it means every frame was already on disk from an earlier run.

UNLISTED is the one to look out for. A FAIL is a single missing frame, but an UNLISTED scan means we do not know how many of its frames are missing — which is why a run can report all its frames present and still be incomplete. Re-running picks up both.

The summary line ends with the reason when the run stopped for one, such as the disk filling up.

Run as a container

Prefer a container (e.g. a pipeline step) over a pip install? The same CLI is published to GHCR:

ghcr.io/salk-harnessing-plants-initiative/bloomctl

docker run --rm ghcr.io/salk-harnessing-plants-initiative/bloomctl:staging \
  cyl ingest-result path/to/scan.result.json

Tags: :staging (latest staging build) · :<version> (matches the PyPI release of the same name) · :sha-<git-sha> (immutable, one per commit). Image provenance and build details are in the repo docs.

Notes

• Species selector — --species NAME filters by species (typed, scriptable); --species-menu picks from a menu. Same on every command; the two are mutually exclusive.
• Interactive menus (--species-menu, --experiment) need a terminal; in a pipe/CI they abort rather than guess. For scripting, pass the typed value/id and use --output json.
• Read vs write — browsing/downloading works for any account; the write commands (ingest-result, datasets create) need an account with write access.
• -n/--workers — how many frames cyl download fetches at once. Default 8, maximum 64, 1 for one at a time. Large experiments run tens of thousands of frames, and this is what makes them quick. More is not always better: if the server starts refusing requests you will see frames fail, and the fix is a lower number, not a higher one.
• Resuming — a download that stops for any reason (interrupted, connection dropped, failed frames) picks up where it left off when you re-run the same command in the same directory. Frames already on disk are skipped. One experiment per output directory.

Documentation

Full docs — per-command detail, the container image, and access roles — are in the project repository.

Tutorials

Download an experiment, from login to files

A start-to-finish walkthrough for the common task — "get me the images + metadata for the soybean drought experiment."

# 1. Log in (once). Defaults to prod; use --server + -p for a named staging/local profile.
bloomctl login
#    → prompts for email + password; writes credentials to ~/.bloom

# 2. Find the experiment — browse by species from a menu (menu prints to stderr):
bloomctl cyl experiments list --species-menu
#    Select a species:
#      0) All species
#      1) Arabidopsis
#      2) Soybean
#    → prints the table; note the experiment_id you want, e.g. 42
#    (or skip this and let `download` resolve the name — see step 4)

# 3. (optional) Sanity-check the contents before pulling gigabytes of images:
bloomctl cyl accessions list --experiment-id 42       # which accessions are in it
bloomctl cyl accessions sample-counts --species-menu  # plant count per accession (pick a species)
bloomctl cyl datasets list --experiment-id 42         # any trait datasets already built

# 4. Download it — metadata only first to preview, then the full pull:
bloomctl cyl download ./soy-drought --experiment-id 42 --meta-only   # scans.csv only
bloomctl cyl download ./soy-drought --experiment-id 42               # scans.csv + all frames
#    …or without ever looking up the id:
bloomctl cyl download ./soy-drought --experiment-name "drought" --species Soybean

Result: ./soy-drought/scans.csv (one row per scan) plus ./soy-drought/images/Wave{n}/… (the frames). For scripting, swap the menus for explicit ids and add --output json.