slurm-helper-utils 0.2.1

SLURM Helper Utils

Collection of Python scripts for SLURM related tasks.

The following exported scripts are available: - create-slurm-script - validate-slurm-script

These scripts expect that the configuration files are present in the current working directory. Recommended steps: 1. Create a directory conf e.g.: mkdir conf. 2. Copy the config.yaml from your virtual environment into that directory e.g.: cp venv/lib/python3.10/site-packages/slurm_helper_utils/conf/config*.yaml conf/.. 3. Edit the configuration files per your needs.

The following build configuration file content will provide the qualified options that the user will interactively select.

cat -n conf/config_builder.yaml
     1  ---
     2  required_directives:
     3    job-name:
     4      max_length: 20
     5      min_length: 5
     6    nodes:
     7      qualified_options:
     8        - 1
     9        - 2
    10        - 3
    11        - 4
    12    qos:
    13      qualified_options:
    14        - qos-A
    15        - qos-B
    16    gres:
    17      qualified_options:
    18        - gres-1
    19        - gres-2
    20    export:
    21      qualified_options:
    22        - NONE
    23    propagate:
    24      qualified_options:
    25        - ALL
    26        - NONE
    27        - AS
    28        - CORE
    29        - CPU
    30        - DATA
    31        - FSIZE
    32        - MEMLOCK
    33        - NOFILE
    34        - NPROC
    35        - RSS
    36        - STACK
    37    mail-user:
    38      qualified_options:
    39        - jay@gmail.com
    40        - sun@gmail.com
    41    mail-type:
    42      qualified_options:
    43        - NONE
    44        - BEGIN
    45        - END
    46        - FAIL
    47        - REQUEUE
    48        - ALL
    49        - INVALID_DEPEND
    50        - STAGE_OUT
    51        - TIME_LIMIT
    52        - TIME_LIMIT_90
    53        - TIME_LIMIT_80
    54        - TIME_LIMIT_50
    55        - ARRAY_TASKS
    56    output:
    57      desc: The STDOUT from the SLURM sbatch script
    58    error:
    59      desc: The STDERR from the SLURM sbatch script
    60    partition:
    61      qualified_options:
    62        - partition-1
    63        - partition-22
    64    ntasks-per-node:
    65      desc: TBD
    66    cpus-per-task:
    67      desc: TBD
    68    time:
    69      desc: TBD

Invoke the exported console script that will allow user to interactively create a SLURM sbatch script:

create-slurm-script --config_file conf/config_builder.yaml
--outdir was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'
Created output directory '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404'
--logfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log'
--outfile was not specified and therefore was set to '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'
There are '14' required directives
Please enter the option for directive 'nodes':
options: ['1', '2', '3', '4']
Enter nodes: 2
Please enter the option for directive 'qos':
options: ['qos-A', 'qos-B']
Enter qos: qos-A
Please enter the option for directive 'gres':
options: ['gres-1', 'gres-2']
Enter gres: gres-1
Please enter the option for directive 'export':
options: ['NONE']
Enter export: NONE
Please enter the option for directive 'propagate':
options: ['ALL', 'NONE', 'AS', 'CORE', 'CPU', 'DATA', 'FSIZE', 'MEMLOCK', 'NOFILE', 'NPROC', 'RSS', 'STACK']
Enter propagate: ALL
Please enter the option for directive 'mail-user':
options: ['jay@gmail.com', 'sun@gmail.com']
Enter mail-user: jay@gmail.com
Please enter the option for directive 'mail-type':
options: ['NONE', 'BEGIN', 'END', 'FAIL', 'REQUEUE', 'ALL', 'INVALID_DEPEND', 'STAGE_OUT', 'TIME_LIMIT', 'TIME_LIMIT_90', 'TIME_LIMIT_80', 'TIME_LIMIT_50', 'ARRAY_TASKS']
Enter mail-type: ALL
Please enter the option for directive 'partition':
options: ['partition-1', 'partition-22']
Enter partition: partition-22
Wrote SLURM sbatch script '/tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh'

Contents of the generated SLURM sbatch script:

cat -n /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/slurm_sbatch.sh
     1  #!/bin/bash
     2
     3  ## method-created: /tmp/test-slurm-helper-utils/venv/lib/python3.10/site-packages/slurm_helper_utils/create_slurm_script.py
     4  ## date-created: 2025-02-15-152508
     5  ## created-by: sundaram
     6  ## config-file: conf/config_builder.yaml
     7  ## logfile: /tmp/sundaram/slurm-helper-utils/create_slurm_script/2025-02-15-152404/create_slurm_script.log
     8
     9
    10  #SBATCH --nodes=2
    11  #SBATCH --qos=qos-A
    12  #SBATCH --gres=gres-1
    13  #SBATCH --export=NONE
    14  #SBATCH --propagate=ALL
    15  #SBATCH --mail-user=jay@gmail.com
    16  #SBATCH --mail-type=ALL
    17  #SBATCH --partition=partition-22

History

0.1.0 (2024-01-09)

• First release on PyPI.