Skip to main content

SlurmGen - Simple Slurm Manager

Project description

SlurmGen - Simple Slurm Manager

Summary

SlurmGen is a very simple Slurm manager:

  • The job description is stored in JSON files.
  • The JSON files are transformed into Slurm scripts.
  • The Slurm script can be run locally or on a Slurm Cluster.

SlurmGen is only supporting a small subset of Slurm:

  • Create and delete folders.
  • Set environment variables.
  • Set the job name and log.
  • Set the ressources (memory, time, CPU, etc.).
  • Define the set of commands to be executed.

SlurmGen is not a binding and/or a wrapper for Slurm. SlurmGen is only generating Slurm scripts from JSON files. SlurmGen is written in Python without any external dependencies.

Example

An example is located in the example folder of the repository:

  • job_def.json contains the job definition
  • job_tmpl.json contains the template data
  • run_slurm.py contains is dummy script executed by the jon
  • slurm_output contains the generated Slurm script and log file
  • data_output contains data generated by the Slurm job
# Create a Slurm script with a command line template.
sgen job_def.json --tmpl_str "title" "test" --tmpl_str "var" "Welcome to everyone"

# Create a Slurm script with a JSON file template.
sgen job_def.json --tmpl_file job_tmpl.json

# Create a Slurm script and run the job locally.
sgen job_def.json --local --tmpl_file job_tmpl.json

# Create a Slurm script and run the job on the cluster.
sgen job_def.json --cluster --tmpl_file job_tmpl.json

Project Links

Author

Copyright

(c) 2023 - Thomas Guillod

BSD 2-Clause "Simplified" License

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slurmgen-3.11.0.tar.gz (10.6 kB view hashes)

Uploaded Source

Built Distribution

slurmgen-3.11.0-py3-none-any.whl (8.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page