HPC Workflows & Edge Service
Project description
This page is for the Balsam 0.6 pre-release. Click here for stable Balsam 0.5 docs.
A unified platform to manage high-throughput workflows across the HPC landscape.
Run Balsam on any laptop, cluster, or supercomputer.
$ pip install --pre balsam-flow
$ balsam login
$ balsam site init my-site
Declare HPC Apps and execution lifecycle hooks.
from balsam.api import ApplicationDefinition
class Hello(ApplicationDefinition):
site = "my-site"
command_template = "echo hello {{ name }}"
def handle_timeout(self):
self.job.state = "RESTART_READY"
Run Apps from anywhere, thanks to the unified Balsam service.
# On any machine with internet access...
from balsam.api import Job, BatchJob
# Create Jobs:
job = Job.objects.create(
site_name="my-site",
app_id="Hello",
workdir="test/say-hello",
parameters={"name": "world!"},
)
# Or allocate resources:
BatchJob.objects.create(
site_id=job.site_id,
num_nodes=1,
wall_time_min=10,
job_mode="serial",
project="local",
queue="local",
)
Define and run Python Apps on heterogeneous resources, from a single session.
import numpy as np
class MyApp(ApplicationDefinition):
site = "theta-gpu"
def run(self, vec):
from mpi4py import MPI
rank = MPI.COMM_WORLD.Get_rank()
print("Hello from rank", rank)
return np.linalg.norm(vec)
jobs = [
MyApp.submit(
workdir=f"test/{i}",
vec=np.random.rand(3),
ranks_per_node=4,
gpus_per_rank=0,
)
for i in range(10)
]
for job in Job.objects.as_completed(jobs):
print(job.workdir, job.result())
Features
- Easy
pip
installation runs out-of-the-box on several HPC systems and is easily adaptable to others. - Balsam Sites are remotely controlled by design: submit and monitor workflows from anywhere
- Run any existing application, with flexible execution environments and job lifecycle hooks
- High-throughput and fault-tolerant task execution on diverse resources
- Define data dependencies for any task: Balsam orchestrates the necessary data transfers
- Elastic queueing: auto-scale resources to the workload size
- Monitoring APIs: query recent task failures, node utilization, or throughput
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
balsam-flow-0.7.0a2.tar.gz
(171.5 kB
view hashes)
Built Distribution
balsam_flow-0.7.0a2-py3-none-any.whl
(206.5 kB
view hashes)
Close
Hashes for balsam_flow-0.7.0a2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a800a39ad111537cb5482923aeffac6b8cb2915a26358fa240764012e26bb4c |
|
MD5 | 3b7524439186d9e399babc6be03936a0 |
|
BLAKE2b-256 | e1b1b8031050e406d2faeb99117a31677ea972fcd4e42620bbbb04ab69fa0962 |