Skip to main content

Slurm Job Pruning

A maintenance tool for canceling stale jobs on Slurm clusters.

A command-line utility for Slurm administrators that identifies and cancels jobs which have been stuck in the PENDING state for longer than a configurable threshold. Affected users are notified by email, with all of their canceled jobs batched into a single message, and a dry-run mode is provided for previewing changes before applying them.

Install and Setup

Install the utility from the CRCD Python repository using pipx:

pipx install crc-prune-stale

Confirm the prune-stale utility is available in your runtime environment:

prune-stale --help

Optionally, create the application log directory to enable persistent file logging. If the directory is not present, or the log file cannot be opened, the application emits a warning and falls back to console-only logging:

mkdir -p /var/log/prune_stale

Logs are written to /var/log/prune_stale/prune_stale.log.

Usage

prune-stale [OPTIONS]

Targeting options

Control which cluster and partitions are queried for jobs.

Option Default Description
--cluster NAME local cluster Cluster name to query for jobs
--partition NAME ... all partitions One or more partition names to query. Omit for all partitions

The default cluster is resolved at startup from the local node's Slurm configuration (scontrol show config). If no cluster name can be determined, the application exits with an error.

Pruning options

Control which jobs are selected for cancellation.

Option Default Description
--threshold DAYS 10 Days a job must have been pending before cancellation
--dry-run off Log which jobs would be cancelled without taking any action

Job age is measured against the job's submit time as reported by squeue. Jobs with malformed or unparsable squeue output are skipped and logged as warnings.

Notification options

Control outbound email notifications.

Option Default Description
--smtp-host HOST — SMTP relay hostname. Omit to disable email notifications
--smtp-port PORT 25 SMTP relay port
--email-from ADDRESS slurm-noreply@crc.pitt.edu Sender address for notification emails
--email-dmn DOMAIN pitt.edu Domain appended to usernames to form recipient addresses

Notifications are only sent when --smtp-host is provided and --dry-run is not set. SMTP failures are logged and do not abort the run.

Pass --help to see all options with their current defaults.

Examples

Cancel all jobs that have been pending for more than 10 days (no email notification):

prune-stale --threshold 10

Cancel jobs and issue an email notification to the job submitter:

prune-stale --threshold 10 --smtp-host mailrelay.pitt.edu

Restrict the run to specific partitions on a specific cluster:

prune-stale --cluster htc --partition standard high-mem

Preview what would be canceled without making any changes:

prune-stale --dry-run

Metadata

Release files for crc-prune-stale 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for crc-prune-stale 0.3.0
File Size Uploaded Provenance
crc_prune_stale-0.3.0.tar.gz 10.8 kB Without provenance Details

Built distribution (wheel)

Table of built distributions (wheels) for crc-prune-stale 0.3.0
File Interpreter ABI Platform
crc_prune_stale-0.3.0-py3-none-any.whl Python 3 none any Details

Total release size: 24.0 kB

Release files / crc_prune_stale-0.3.0.tar.gz

Download URL crc_prune_stale-0.3.0.tar.gz
Size 10.8 kB
Tags Source
SHA-256 checksum
How to use checksums
0ad47534a526a6f57e30a5ab86041212a1b03971fb9fbd1f1c75b375811498bf
BLAKE2b-256 checksum
How to use checksums
07b5760f44c116252c598acd03f222c3496f2d02231eb257e06a90b1e0099dde
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / crc_prune_stale-0.3.0-py3-none-any.whl

Download URL crc_prune_stale-0.3.0-py3-none-any.whl
Size 13.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
20a528a39a50c7b57665df441086da124ca50816ca4579624f9aab412b114e16
BLAKE2b-256 checksum
How to use checksums
51f70d5ba23fed1b915c2e2e306aa147a1fe67eb4ba71f050c3d60006716e15b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for crc-prune-stale 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page