HPC Workflows & Edge Service
Project description
This page is for the Balsam 0.6 pre-release. Click here for stable Balsam 0.5 docs.
A unified platform to manage high-throughput workflows across the HPC landscape.
Run Balsam on any laptop, cluster, or supercomputer.
$ pip install --pre balsam-flow
$ balsam login
$ balsam site init my-site
Declare HPC Apps and execution lifecycle hooks.
from balsam.api import ApplicationDefinition
class Hello(ApplicationDefinition):
site = "my-site"
command_template = "echo hello {{ name }}"
def handle_timeout(self):
self.job.state = "RESTART_READY"
Run Apps from anywhere, thanks to the unified Balsam service.
# On any machine with internet access...
from balsam.api import Job, BatchJob
# Create Jobs:
job = Job.objects.create(
site_name="my-site",
app_id="Hello",
workdir="test/say-hello",
parameters={"name": "world!"},
)
# Or allocate resources:
BatchJob.objects.create(
site_id=job.site_id,
num_nodes=1,
wall_time_min=10,
job_mode="serial",
project="local",
queue="local",
)
Define and run Python Apps on heterogeneous resources, from a single session.
import numpy as np
class MyApp(ApplicationDefinition):
site = "theta-gpu"
def run(self, vec):
from mpi4py import MPI
rank = MPI.COMM_WORLD.Get_rank()
print("Hello from rank", rank)
return np.linalg.norm(vec)
jobs = [
MyApp.submit(
workdir=f"test/{i}",
vec=np.random.rand(3),
ranks_per_node=4,
gpus_per_rank=0,
)
for i in range(10)
]
for job in Job.objects.as_completed(jobs):
print(job.workdir, job.result())
Features
- Easy
pip
installation runs out-of-the-box on several HPC systems and is easily adaptable to others. - Balsam Sites are remotely controlled by design: submit and monitor workflows from anywhere
- Run any existing application, with flexible execution environments and job lifecycle hooks
- High-throughput and fault-tolerant task execution on diverse resources
- Define data dependencies for any task: Balsam orchestrates the necessary data transfers
- Elastic queueing: auto-scale resources to the workload size
- Monitoring APIs: query recent task failures, node utilization, or throughput
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
balsam-flow-0.7.0a1.tar.gz
(171.1 kB
view hashes)
Built Distribution
balsam_flow-0.7.0a1-py3-none-any.whl
(206.0 kB
view hashes)
Close
Hashes for balsam_flow-0.7.0a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7816dc52bda1107cc15d5e62f00a2af2fd759929ebe1e3fd33f0f5fd92d492f6 |
|
MD5 | ff48cf537d19d14ed0523a34bb8f7520 |
|
BLAKE2b-256 | a2767c25c1b9d170261be220a273105a58856f63b2af0b7259c50c5160fbb38f |