Jupyter slurm provisioner
Installation
pip install slurm-provisioner
Overview
The slurm provisioner is a subclass of `jupyter_client.KernelProvisionerBase <https://github.com/jupyter/jupyter_client/blob/v7.4.2/jupyter_client/provisioning/provisioner_base.py#L24>`__. Its area of use is any slurm-based HPC system. However, it was written to be used within the Juelich Supercomputing Centre, which uses a script called jutil to create a user-specific environment based on a project. You may have to update scripts/slurmel_allocate to use it on any other HPC system. It allows users to start their jupyter kernel on any partition, while their notebook server is running on an interactive login node. This offers a separation between code execution and user interface. The slurm provisioner will not use its ipykernel argv configuration but is meant to be a wrapper for any existing kernel. It is recommended to use this kernel with the option `--KernelRestarter.restart_limit=0 <https://github.com/jupyter/jupyter_client/blob/v7.4.2/jupyter_client/restarter.py#L43>`__ to avoid unsought compute time and costs.
Configuration
Configure a kernel.json file in your $HOME. It is not recommended to define the kernel globally, since the configuration is user-specific.
Options
project [required]: Used for correct budgeting of compute time.
partition [required]: specify the slurm partition
nodes [required]: specify the number of nodes
runtime [required]: specify the runtime in minutes
kernel_argv [required]: copy & paste this from the kernel you want to run
gpus [optional]: specify the number of GPUs, if the partition supports GPUs
reservation [optional]: specify the slurm reservation, if you have one
jobid [optional]: specify a pre-existing slurm allocation and start your kernel there. Without this, a new allocation for each kernel will be created.
node [optional]: specify a node in your pre-existing allocation
Example kernel.json
.local/share/jupyter/kernels/slurmel/kernel.json
{
"display_name": "Slurm Kernel",
"language": "python",
"metadata": {
"debugger": true,
"kernel_provisioner": {
"config": {
"gpus": "0",
"nodes": "2",
"partition": "batch",
"project": "...",
"reservation": "None",
"runtime": 30,
"jobid": "None",
"node": "None",
"kernel_argv": [
"~/.local/share/jupyter/kernels/my_kernel/kernel.sh",
"-f",
"{connection_file}"
]
},
"provisioner_name": "slurm-provisioner"
}
}
}
Metadata
Release files for jupyter-slurm-provisioner 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| jupyter-slurm-provisioner-0.6.0.tar.gz | 11.7 kB | Without provenance | Details for jupyter-slurm-provisioner-0.6.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| jupyter_slurm_provisioner-0.6.0-py3-none-any.whl | Python 3 | none | any | Details for jupyter_slurm_provisioner-0.6.0-py3-none-any.whl |
Total release size: 23.5 kB
Release files / jupyter-slurm-provisioner-0.6.0.tar.gz
| Download URL | jupyter-slurm-provisioner-0.6.0.tar.gz |
|---|---|
| Size | 11.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
24141dc4abfb009d424837a5d78159c2b63e0243d63618cacebcf4f23edc101f
|
|
BLAKE2b-256 checksum How to use checksums |
1af71336205f53e79ffb5e7acbac42be2656cb2fe43fe750b561a28150213254
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.11.0
|
Release files / jupyter_slurm_provisioner-0.6.0-py3-none-any.whl
| Download URL | jupyter_slurm_provisioner-0.6.0-py3-none-any.whl |
|---|---|
| Size | 11.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
73d454aa68cbb5e9a018bab5e2b3460901b6e78e47a32fb695c44ab6fd3d6f98
|
|
BLAKE2b-256 checksum How to use checksums |
58333444e9a83a2744eb9b7b65a06969d688e1e48ef3823b0988d3de048e3af1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.11.0
|
Security for jupyter-slurm-provisioner 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.