snakemake-executor-plugin-sge

A Snakemake executor plugin for Sun Grid Engine (SGE), Univa Grid Engine (UGE), and Open Grid Scheduler (OGS) clusters.

Features

Every job is an array job – singletons become -t 1-1, batches become -t 1-N. One submission path, one tracking path, no special cases.
Group jobs as a single array job – Snakemake group jobs (several rules bundled together) are submitted as one qsub call.
Rich qsub flag coverage – queue, project, parallel environment, wall time, memory (per-slot and total), priority, requeue, reservation, notify, mail, hold_jid, task concurrency, env-var export, join logs, and arbitrary extra flags.
Per-rule resource overrides – any global setting can be overridden via rule resources: (e.g. sge_queue, sge_pe, sge_extra, …).
Two-stage status polling – qstat for in-flight jobs, qacct for completed jobs; configurable retries and optional qacct disabling.
Clean log management – per-task stdout/stderr, optional auto-delete on success, age-based rotation.

Installation

pip install snakemake-executor-plugin-sge

Quick start

snakemake --executor sge --jobs 100

Common options

CLI flag	Default	Description
`--sge-queue`	—	Default queue (`-q`)
`--sge-pe`	—	Parallel environment for multi-thread jobs
`--sge-project`	—	Project (`-P`)
`--sge-export-env`	`True`	Export environment (`-V`)
`--sge-join-logs`	`False`	Merge stdout+stderr (`-j y`)
`--sge-requeue`	—	Requeue on failure (`-r y/n`)
`--sge-reservation`	—	Resource reservation (`-R y/n`)
`--sge-notify`	`False`	Send notify signal
`--sge-mail-on`	—	Mail events (`b`, `e`, `a`, `s`, `n`)
`--sge-mail-address`	—	Mail address (`-M`)
`--sge-hold-jid`	—	Hold until job(s) finish
`--sge-array-limit`	`75000`	Max tasks per `qsub -t` call
`--sge-task-concurrency`	—	Max concurrent array tasks (`-tc`)
`--sge-logdir`	`.snakemake/sge_logs`	Log directory
`--sge-keep-successful-logs`	`False`	Keep logs for successful jobs
`--sge-use-qacct`	`True`	Use `qacct` for finished-job detection
`--sge-extra`	—	Raw extra qsub flags (global)
`--sge-jobname-prefix`	—	Prefix for SGE job names

Per-rule resource overrides

rule my_rule:
    resources:
        runtime        = 120,          # minutes → h_rt=02:00:00
        mem_mb         = 8192,         # total memory (divided by threads per slot)
        mem_mb_per_cpu = 2048,         # per-slot memory (takes precedence)
        threads        = 4,
        sge_queue      = "highmem.q",
        sge_pe         = "smp",
        sge_project    = "myproject",
        sge_extra      = "-l gpu=1",   # arbitrary extra qsub flags
        sge_resources  = {"h_cpu": "24:00:00", "arch": "lx-amd64"},
        sge_join_logs  = True,
        sge_requeue    = True,
        sge_priority   = -100,
        sge_notify     = True,
        sge_mail_on    = "e",
        sge_mail_address = "me@example.com",
        sge_hold_jid   = "12345",
        sge_task_concurrency = 8,
        sge_export_env = False,
    threads: 4
    shell: "..."

Array-job design

Every qsub submission uses qsub -t start-end. Execution commands are stored in a JSON file on the shared filesystem before qsub is called. The submission script reads $SGE_TASK_ID, looks up the compressed command, and evaluates it. This avoids command-line length limits and all heredoc/quoting problems.

License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

snakemake_executor_plugin_sge-0.6.22.tar.gz (25.2 kB view details)

Uploaded Jul 8, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

snakemake_executor_plugin_sge-0.6.22-py3-none-any.whl (26.3 kB view details)

Uploaded Jul 8, 2026 Python 3

File details

Details for the file snakemake_executor_plugin_sge-0.6.22.tar.gz.

File metadata

Download URL: snakemake_executor_plugin_sge-0.6.22.tar.gz
Upload date: Jul 8, 2026
Size: 25.2 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for snakemake_executor_plugin_sge-0.6.22.tar.gz
Algorithm	Hash digest
SHA256	`5c6378338706cc5f3547e91e91bfaed54e1a1e3cfcc8d4fb5c9434c10b097856`
MD5	`66ea028838e1c58dafb3b6119ab85f0a`
BLAKE2b-256	`100f37d62714ad4547066b03660c87ade7ed8b67d69f072207d6ef6c4cb5f062`

See more details on using hashes here.

File details

Details for the file snakemake_executor_plugin_sge-0.6.22-py3-none-any.whl.

File metadata

Download URL: snakemake_executor_plugin_sge-0.6.22-py3-none-any.whl
Upload date: Jul 8, 2026
Size: 26.3 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for snakemake_executor_plugin_sge-0.6.22-py3-none-any.whl
Algorithm	Hash digest
SHA256	`4fcd3a0c41bff8f72cfa060aba91163d2550d4c54a5640d1c648853146df97ba`
MD5	`a140a5b7cfd44285893d13bd476a518f`
BLAKE2b-256	`b5bd4c9e54839036b5c34bf5559ea80f1afdc5f687723da7c45d4451796b1bea`