SLURM utilities for Python
Project description
pyslurmutils
SLURM utilities for Python.
Demo
Get an access token on rnice (prints SLURM_JWT=<token>
)
export SLURM_TOKEN=$(scontrol token lifespan=3600)
Run some example jobs
python3 scripts/example.py
Run the tests (CI or locally)
export SLURM_TOKEN=...
export SLURM_USER=...
python3 -m pytest .
Execute a python function on SLURM
High-level API
API mimics python's concurrent.futures
API
from pyslurmutils.futures import SlurmExecutor
with SlurmExecutor(
url,
user_name,
token,
log_directory=log_directory,
data_directory=data_directory, # TCP when not provided
) as pool:
future = pool.submit(sum, args=([1, 1],), pre_script="conda activate myenv")
assert future.result() == 2
Low-level API
from pyslurmutils.client.pyscript import SlurmPythonJobApi
with SlurmPythonJobApi(
url,
user_name,
token,
log_directory=log_directory,
data_directory=data_directory, # TCP when not provided
) as pyapi:
job_id = pyapi.spawn(sum, args=([1, 1],), pre_script="conda activate myenv")
try:
pyapi.wait_done(job_id)
pyapi.print_stdout_stderr(job_id)
assert pyapi.get_result(job_id) == 2
finally:
pyapi.clean_job_artifacts(job_id)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyslurmutils-0.0.1a0.tar.gz
(13.7 kB
view hashes)