Skip to main content

fast command-line extensions for sourmash

Project description



tl;dr Do fast and low-memory search/gather of many sourmash sketches via a sourmash plugin.


This repo contains a PyO3-based Python wrapper around the core branchwater code. Branchwater is a fast, low-memory and multithreaded application for searching very large collections of FracMinHash sketches as generated by sourmash.

For details, see the Rust code in src/ and Python wrapper in src/python/.

Uses pyo3 for the Python-to-Rust wrapping.

This functionality can be used from within sourmash as a command-line plugin; see below quickstart.


There is a quickstart below, as well as more documentation here.

Quickstart for manysearch.

To try out branchwater, you'll need to install sourmash 4.8.3 or later.

This quickstart demonstrates multisearch using the 64 genomes from Awad et al., 2017.

1. Install necessary dependencies

You'll need rust, Python, and maturin to build, and sourmash to run. See environment.yml for a list of conda packages, and developer docs below for example command lines.

2. Install sourmash_plugin_branchwater.

Install this repo in developer mode:

pip install -e .

3. Download sketches.

The following commands will download sourmash sketches for the podar genomes into the file

curl -L -o

5. Execute!

Now run multisearch to search all the sketches against each other:

sourmash scripts multisearch -o results.csv --cores 4

You will (hopefully ;)) see a set of results in results.csv. These are comparisons of each query against all matching genomes.

Debugging help

If your collections aren't loading properly, try running sourmash sig summarize on them, like so:

sourmash sig summarize

This will make sure everything can be loaded properly.

Future thoughts

The speed and functions of this code will probably be brought into sourmash core in the future, most likely as part of sourmash#2230. However, in the meantime, this is a fun and useful side project that makes use of sourmash plugins and Rust to provide some fast functionality, and it can serve as a testbed for future sourmash functionality.

Developer notes

Installing a development environment

You'll need sourmash, rust, and maturin.

A simple way to get up and running is to run:

mamba env create -n branchwater-dev -f environment.yml

in the top directory of the repo, and then activate the environment and install in editable mode:

mamba activate branchwater-dev
pip install -e .

Running the tests locally


make test

will run the Python tests.

Generating a release

  1. Bump version number in Cargo.toml and run make to update Cargo.lock. Then commit and push to origin/main.

  2. Make a new release on github with a matching version tag.

  3. Then pull, and:

make sdist
make upload_sdist

to create a new release on PyPI.

Building wheels

You can build a release wheel for your current platform with:

make wheel

and it will be placed under target/wheels/.


This software is under the AGPL license. Please see LICENSE.txt.


  • Luiz Irber
  • C. Titus Brown
  • Mohamed Abuelanin
  • N. Tessa Pierce-Ward

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sourmash_plugin_branchwater-0.8.4.tar.gz (22.1 MB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page