Skip to main content

submit jobs to slurm with python

Project description

quick and dirty lib for submitting jobs to slurm via python2/python3.

from slurmpy import Slurm

s = Slurm("job-name", {"account": "my-account", "partition": "my-parition"})
s.run("""
do
lots
of
stuff
""")

The above will submit the job to sbatch automatically write the script to scripts/ and automatically write logs/{name}.err and logs/{name}.out. It will have today's date in the log and script names.

The script to run() can also contain $variables which are filled with the cmd_kwarg dict. E.g. echo $name could be filled with cmd_kwargs={'name': 'sally'}

A command can be tested (not sent to queue) by setting the _cmd are to run as e.g. "ls". The default is sbatch which submits jobs to slurm.

Dependencies

Each time slurmpy.Slurm().run() is called, it returns the job-id of the submitted job. This can then be sent to a subsequent job:

s = Slurm()
s.run(..., depends_on=[job_id])

to indicate that this job should not run until the the job with job_id has finished successfully.

Install

pip install slurmpy --user

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slurmpy-0.0.8.tar.gz (3.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page