IKernel Remote

All your Jupyter kernels, on all your machines, in one place. This is a fork of abandonned package remote_ikernel by Tom Daff <tdd20@cam.ac.uk>.

Launch Jupyter kernels on remote systems and through batch queues so that they can be used within a local Jupyter noteboook.

Jupyter compatible Kernels start through interactive jobs in batch queue systems (SGE, SLURM, PBS…) or through SSH connections. Once the kernel is started, SSH tunnels are created for the communication ports are so the notebook can talk to the kernel as if it was local.

Commands for managing the kernels are included. It is also possible to use ikr script to manage kernels from different virtual environments or different python implementations.

Install with pip install ikernel_remote. Requires notebook (as part of Jupyter), version 4.0 or greater and pexpect. Passwordless ssh to the all the remote machines is also required (e.g. nodes on a cluster).

# Install the module ('python setup.py install' also works)

pip install ikernel_remote

# Set up the kernels you'd like to use

ikr

# Add a new kernel running through GrideEngine

ikr --add \
   --kernel_cmd="ipython kernel -f {connection_file}" \
   --name="Python 2.7" --cpus=2 --pe=smp --interface=sge

# Add an SSH connection to a remote machine running IJulia

ikr --add \
   --kernel_cmd="/home/me/julia-79599ada44/bin/julia -i -F /home/me/.julia/v0.3/IJulia/src/kernel.jl {connection_file}" \
   --name="IJulia 0.3.8" --interface=ssh \
   --host=me@remote.machine --workdir='/home/me/Workdir' --language=julia

# Set up kernels for your local virtual environments that can be run
# from a single notebook server.

ikr --add \
   --kernel_cmd="/home/me/Virtualenvs/dev/bin/ipython kernel -f {connection_file}" \
   --name="Python 2 (venv:dev)" --interface=local

# Connect to a SLURM cluster through a gateway machine (to get into a
# local network) and cluster frontend machine (where the sqsub runs from).

ikr --add \
   --kernel_cmd="ipython kernel -f {connection_file}" \
   --name="Python 2.7" --cpus=4 --interface=slurm \
   --tunnel-hosts gateway.machine cluster.frontend

The kernel spec files will be installed so that the new kernel appears in the drop-down list in the notebook. ikernel_remote manage also has options to show and delete existing kernels.