Skip to main content

submit jobs to slurm with python

Project description

quick and dirty lib for submitting jobs to slurm via python2/python3.

from slurmpy import Slurm

s = Slurm("job-name", {"account": "my-account", "partition": "my-parition"})
s.run("""
do
lots
of
stuff
""")

The above will submit the job to sbatch automatically write the script to scripts/ and automatically write logs/{name}.err and logs/{name}.out. It will have today's date in the log and script names.

The script to run() can also contain $variables which are filled with the cmd_kwarg dict. E.g. echo $name could be filled with cmd_kwargs={'name': 'sally'}

A command can be tested (not sent to queue) by setting the _cmd are to run as e.g. "ls". The default is sbatch which submits jobs to slurm.

Dependencies

Each time slurmpy.Slurm().run() is called, it returns the job-id of the submitted job. This can then be sent to a subsequent job:

s = Slurm()
s.run(..., depends_on=[job_id])

to indicate that this job should not run until the the job with job_id has finished successfully.

Install

pip install slurmpy --user

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slurmpy-0.0.8.tar.gz (3.8 kB view details)

Uploaded Source

File details

Details for the file slurmpy-0.0.8.tar.gz.

File metadata

  • Download URL: slurmpy-0.0.8.tar.gz
  • Upload date:
  • Size: 3.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.1.post20200322 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.6

File hashes

Hashes for slurmpy-0.0.8.tar.gz
Algorithm Hash digest
SHA256 17da07cb9b46cb02fa21cc6d076f6baac6f5b351b79fb1a1f23a857b6161e470
MD5 f62b68ca492917b2aa3fd909dd6ee1a7
BLAKE2b-256 6ecf58c9e6575affc4fc7fb9a0cd0b246b612898b621650f8734770c33c52e35

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page