hypha-launcher 0.1.1

Launcher for hypha services

hypha-launcher

Run triton server on HPC

Work In Progress

Features

• CLI/API for:
  • downloading model from s3 and pulling docker image of triton server
  • launch s3 server
  • launch triton server
  • ...
• Support different container engines
  • Docker
  • Apptainer
• Support different compute environments
  • Local
  • Slurm

Installation

pip install hypha-launcher

CLI Usage

$ hypha-launcher --help

Launch the bioimage.io backend

Download all models from s3 and launch triton server.

$ python -m hypha_launcher launch_bioimageio_backend --service-id my-triton

Launch on slurm cluster.

# Please replace the slurm settings with your own settings
$ export SLURM_ACCOUNT=Your-Slurm-Account
$ export SLURM_TIME=03:00:00
$ export SLURM_GPUS_PER_NODE=A100:1
$ python -m hypha_launcher launch_bioimageio_backend --service-id my-triton

Download model from s3

$ python -m hypha-launcher - download_models_from_s3 bioengine-model-runner.* --n_parallel=5

Pull docker image of triton server

$ python -m hypha-launcher - pull_image

TODO

• Download model from s3
• Pull docker image of triton server
• Run triton server
• Register service on hypha
• Conmmunicate with triton server
• Test on HPC
• Support run on local machine without GPU
• Support launch containers inside a container (For support run inside the podman-desktop)
• Job management(Auto stop and restart)
• Load balancing
• Documentation

Development

Install the package in editable mode with the following command:

pip install -e .
pip install -r requirements-dev.txt