Skip to main content

Jupyter slurm provisioner

Installation

pip install slurm-provisioner

Overview

The slurm provisioner is a subclass of `jupyter_client.KernelProvisionerBase <https://github.com/jupyter/jupyter_client/blob/v7.4.2/jupyter_client/provisioning/provisioner_base.py#L24>`__. Its area of use is any slurm-based HPC system. However, it was written to be used within the Juelich Supercomputing Centre, which uses a script called jutil to create a user-specific environment based on a project. You may have to update scripts/slurmel_allocate to use it on any other HPC system. It allows users to start their jupyter kernel on any partition, while their notebook server is running on an interactive login node. This offers a separation between code execution and user interface. The slurm provisioner will not use its ipykernel argv configuration but is meant to be a wrapper for any existing kernel. It is recommended to use this kernel with the option `--KernelRestarter.restart_limit=0 <https://github.com/jupyter/jupyter_client/blob/v7.4.2/jupyter_client/restarter.py#L43>`__ to avoid unsought compute time and costs.

Configuration

Configure a kernel.json file in your $HOME. It is not recommended to define the kernel globally, since the configuration is user-specific.

Options

  • project [required]: Used for correct budgeting of compute time.

  • partition [required]: specify the slurm partition

  • nodes [required]: specify the number of nodes

  • runtime [required]: specify the runtime in minutes

  • kernel_argv [required]: copy & paste this from the kernel you want to run

  • gpus [optional]: specify the number of GPUs, if the partition supports GPUs

  • reservation [optional]: specify the slurm reservation, if you have one

  • jobid [optional]: specify a pre-existing slurm allocation and start your kernel there. Without this, a new allocation for each kernel will be created.

  • node [optional]: specify a node in your pre-existing allocation

Example kernel.json

.local/share/jupyter/kernels/slurmel/kernel.json

{
    "display_name": "Slurm Kernel",
    "language": "python",
    "metadata": {
        "debugger": true,
        "kernel_provisioner": {
            "config": {
                "gpus": "0",
                "nodes": "2",
                "partition": "batch",
                "project": "...",
                "reservation": "None",
                "runtime": 30,
                "jobid": "None",
                "node": "None",
                "kernel_argv": [
                    "~/.local/share/jupyter/kernels/my_kernel/kernel.sh",
                    "-f",
                    "{connection_file}"
                ]
            },
            "provisioner_name": "slurm-provisioner"
        }
    }
}

Metadata

Release files for jupyter-slurm-provisioner 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for jupyter-slurm-provisioner 0.6.0
File Size Uploaded Provenance Actions
jupyter-slurm-provisioner-0.6.0.tar.gz 11.7 kB Without provenance Details for jupyter-slurm-provisioner-0.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for jupyter-slurm-provisioner 0.6.0
File Interpreter ABI Platform Actions
jupyter_slurm_provisioner-0.6.0-py3-none-any.whl Python 3 none any Details for jupyter_slurm_provisioner-0.6.0-py3-none-any.whl

Total release size: 23.5 kB

Release files / jupyter-slurm-provisioner-0.6.0.tar.gz

Download URL jupyter-slurm-provisioner-0.6.0.tar.gz
Size 11.7 kB
Tags Source
SHA-256 checksum
How to use checksums
24141dc4abfb009d424837a5d78159c2b63e0243d63618cacebcf4f23edc101f
BLAKE2b-256 checksum
How to use checksums
1af71336205f53e79ffb5e7acbac42be2656cb2fe43fe750b561a28150213254
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.11.0

Release files / jupyter_slurm_provisioner-0.6.0-py3-none-any.whl

Download URL jupyter_slurm_provisioner-0.6.0-py3-none-any.whl
Size 11.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
73d454aa68cbb5e9a018bab5e2b3460901b6e78e47a32fb695c44ab6fd3d6f98
BLAKE2b-256 checksum
How to use checksums
58333444e9a83a2744eb9b7b65a06969d688e1e48ef3823b0988d3de048e3af1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.11.0

Security for jupyter-slurm-provisioner 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page