Skip to main content

Utilities and APIs for interfacing with the Slurm workload manager.

Project description

slurmutils

PyPI - Version PyPI - Downloads GitHub License Matrix

Utilities and APIs for interfacing with the Slurm workload manager.

slurmutils is a collection of various utilities that make it easier for you and your friends to interface with the Slurm workload manager, especially if you are orchestrating deployments of new and current Slurm clusters. Gone are the days of seething over incomplete Jinja2 templates. Current utilities shipped in the slurmutils package include:

from slurmutils.editors import ...

  • slurmconfig: An editor for slurm.conf configuration files.
  • slurmdbdconfig: An editor for slurmdbd.conf configuration files.
  • cgroupconfig: An editor for cgroup.conf configuration files.

For more information on how to use or contribute to slurmutils, check out the Getting Started and Development sections below 👇

✨ Getting Started

Installation

Option 1: Install from PyPI

$ python3 -m pip install slurmutils

Option 2: Install from source

We use the Poetry packaging and dependency manager to manage this project. It must be installed on your system if installing slurmutils from source.

$ git clone https://github.com/canonical/slurmutils.git
$ cd slurmutils
$ poetry install

Usage

slurmutils.editors

This module provides an API for editing files, and creating new files if they do not exist. Here's some operations you can perform on files using the editors in this module:

slurmconfig
Edit a pre-existing slurm.conf configuration file
from slurmutils.editors import slurmconfig

# Open, edit, and save the slurm.conf file located at _/etc/slurm/slurm.conf_.
with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
    del config.inactive_limit
    config.max_job_count = 20000
    config.proctrack_type = "proctrack/linuxproc"
Add a new node to the slurm.conf file
from slurmutils.editors import slurmconfig
from slurmutils.models import Node

with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
    node = Node(
        NodeName="batch-[0-25]", 
        NodeAddr="12.34.56.78", 
        CPUs=1, 
        RealMemory=1000, 
        TmpDisk=10000,
    )
    config.nodes.update(node.dict())
slurmdbdconfig
Edit a pre-existing slurmdbd.conf configuration file
from slurmutils.editors import slurmdbdconfig

with slurmdbdconfig.edit("/etc/slurm/slurmdbd.conf") as config:
    config.archive_usage = "yes"
    config.log_file = "/var/spool/slurmdbd.log"
    config.debug_flags = ["DB_EVENT", "DB_JOB", "DB_USAGE"]
    del config.auth_alt_types
    del config.auth_alt_parameters
cgroupconfig
from slurmutils.editors import cgroupconfig

with cgroupconfig.edit("/etc/slurm/cgroup.conf") as config:
    config.constrain_cores = "yes"
    config.constrain_devices = "yes"
    config.constrain_ram_space = "yes"
    config.constrain_swap_space = "yes"

🤔 What's next?

If you want to learn more about all the things you can do with slurmutils, here are some further resources for you to explore:

🛠️ Development

This project uses tox as its command runner, which provides some useful commands that will help you while hacking on slurmutils:

tox run -e fmt   # Apply formatting standards to code.
tox run -e lint  # Check code against coding style standards.
tox run -e unit  # Run unit tests.

If you're interested in contributing your work to slurmutils, take a look at our contributing guidelines for further details.

🤝 Project and community

slurmutils is a project of the Ubuntu High-Performance Computing community. Interested in contributing bug fixes, new editors, documentation, or feedback? Want to join the Ubuntu HPC community? You’ve come to the right place 🤩

Here’s some links to help you get started with joining the community:

📋 License

slurmutils is free software, distributed under the GNU Lesser General Public License, v3.0. See the LGPL-3.0 LICENSE file for further details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

slurmutils-0.8.3.tar.gz (17.6 kB view details)

Uploaded Source

Built Distribution

slurmutils-0.8.3-py3-none-any.whl (25.2 kB view details)

Uploaded Python 3

File details

Details for the file slurmutils-0.8.3.tar.gz.

File metadata

  • Download URL: slurmutils-0.8.3.tar.gz
  • Upload date:
  • Size: 17.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.12.3 Linux/6.8.0-48-generic

File hashes

Hashes for slurmutils-0.8.3.tar.gz
Algorithm Hash digest
SHA256 6f2c23aab6d956243adb3a00a5853dd182b954773f54a5a34b66e1302a5602ba
MD5 6c8778b52f71706d66e3608e30eead48
BLAKE2b-256 7b169e3ee5ca046a464bec4be89a453cbe130623658064beedca42f6d778bec9

See more details on using hashes here.

File details

Details for the file slurmutils-0.8.3-py3-none-any.whl.

File metadata

  • Download URL: slurmutils-0.8.3-py3-none-any.whl
  • Upload date:
  • Size: 25.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.12.3 Linux/6.8.0-48-generic

File hashes

Hashes for slurmutils-0.8.3-py3-none-any.whl
Algorithm Hash digest
SHA256 e1dcd156920289ac16f9fe79da3308f9d06c306660cbeae4cfe63071ddea1df4
MD5 9b18fdc5b9127811a171fc0b112b3534
BLAKE2b-256 3ae93c7c5272e5b90b1059d8fffa91c7c767df471e39f65430fe1fcabcf2d1eb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page