remote jupyter kernels via ssh tunnels
Project description
The design of this pakage is based upon SSH Kernel which is
in turn based upon remote_ikernel. This implementation shares
the same command line parameters as SSH Kernel, but it was
reimplemented from scratch to support Python 3.10. It also includes an ls
implementation which allows
checking on the available kernel specifications.
While there are modest additions to SSH Kernel, there are also modest subtractions. There are fewer configuration options for things like the name that Jupyter Client uses to refer to the kernel. This is still based on the kernel description that the user sees, but the entire name is no longer completely configurable.
Listing the Jupyter Kernels that are available
It can be difficult to know which Jupyter Kernels are available because there is more than one location that
the Kernel Spec files can be
stored. sshpyk
has an ls
option to all for seeing which kernels are available (even if they are
not SSH Kernel or sshpyk
kernels:
bash$ bash$ python -m sshpyk.kernel.ls --no-check -a python3 /Users/drs/develop/python/conda/envs/py310/share/jupyter/kernels/python3 python3.8 /usr/local/share/jupyter/kernels/python3.8 python3dbg /Users/drs/Library/Jupyter/kernels/python3dbg ssh__sshhost06test001 /Users/drs/Library/Jupyter/kernels/ssh__sshhost06test001 ssh__sshhost06test002 /Users/drs/Library/Jupyter/kernels/ssh__sshhost06test002 ssh_host06_host06homehost06condaenvspy310 /Users/drs/Library/Jupyter/kernels/ssh_host06_host06homehost06condaenvspy310 bash$
The --no-check
(or alternatively -nc
) flag indicates that the validity of the kernel spec files
should not be checked. The -a
(or --all
) flag indicates that it should show all kernel
specifications rather than just the ones for SSH Kernel or
sshpyk
kernel specification files.
If --no-check
is not supplied, then as part of listing the kernel information sshpyk
will
verify that the Python executable specified in the kernel specification exist on the local and remote systems.
--local
will limit the check to just the local Python executable and --remote
will limit the
check to only the remote Python executable. These options also list the local or remote Python path instead
of the path to the kernel specification directory.
Command line “ls” options
The following options are available for listing the Jupyter kernel specifications:
–help, -h
Show help information and exit.
–all, -a
List all kernels that are available rather than just SSH Kernel and sshpyk kernels.
–local, -l
Only list the information for the local Python executable.
–remote, -r
Only list the information for the remote Python executable.
–no-check, -nc
Do not check for the existence local or remote Python executables. This option can be used alone or with other options, e.g. with
-l
.--no-check
avoids colorization to indicate problems so it can be useful for scripting.
Adding a new Kernel for a Remote System
sshpyk
can also be used to add a specification file for a remote Python Kernel. For a
remote kernel to work
ssypyk
must be installed on the local systemipykernel
must be installed on the remote system
Once these requirements are satisfied, the new kernel can be added like:
bash$ python3 -m sshpyk.kernel.add --host host06 --python /home/host06/conda/envs/py310 --display-name 'host06 kernel'
This will add a Python kernel which will run on host06
, and it will be called
host06 kernel
when it is listed as an option for the user. We can check to see if it
is now included by using the ls
functionality:
bash$ python3 -m sshpyk.kernel.ls -r -a -nc python3 localhost:python python3.8 localhost:/opt/local/bin/python3.8 python3dbg localhost:/Users/drs/develop/casagui-ic-debugging/iclean_demo_venv/bin/python ssh__sshhost06test001 host06:/home/host06/conda/envs/py310/bin/python ssh__sshhost06test002 host06:/home/host06/conda/envs/py310/bin/python ssh_host06_host06homehost06condaenvspy310 host06:/home/host06/conda/envs/py310/bin/python ssh_host06_host06kernel host06:/home/host06/conda/envs/py310/bin/python bash$
Unlike the example above, here we have asked that the remote Python path be displayed
instead of showing the kernel specificaton directory. Since we again asked that all kernels
be displayed instead of only the SSH kernels, a Python path is displayed for the
non-SSH kernels, but it is the local Python path as indicated by localhost:
.
Because these three kernels are non-SSH kernels this is the only Python path that is
available. However for the SSH kernels, we can see the remote Pyton path listed.
These paths are prefixed with the hostname, here host06:
. We can also see
the newly added kernel listed as ssh_host06_host06kernel
. This name is an internal
name created from the --display-name
string which the end user will typically
never see.
Command line “add” options
–help, -h
Show help information and exit.
–timeout TIMEOUT, -t TIMEOUT
Specify timeout to wait for kernel startup text. This option is not used by
sshpyk
. It is only used byssh_ipykernel
.
–env [ENV …], -e [ENV …]
Specify environment variables for access by code executed within the remote Python kernel the form:
"NAME=VALUE"
–display-name DISPLAY_NAME, -d DISPLAY_NAME
Specify the string to be used to describe this kernel to the end user.
–sudo, -s
Use
sudo
to start kernel on the remote machine. This option is not currently used bysshpyk
. It is only used byssh_ipykernel
.
–host HOST, -H HOST
The name of remote host as used to connect with SSH.
–python PATH, -p PATH
Path to remote python installation. This is the path to the root of the Python installation so the Python executable would be found in
<PATH>/bin/python
.
SSH configuration notes
The host name used above is different from the Internet Protocol name for hosts which have
a well defined address. It is also a name apart from the physical network where the host can
be found. SSH allows for rationalizing the naming of the hosts to which you have access.
This is done through the $HOME/.ssh/config
file.
The most useful configuration option with respect to sshpyk
is the ability to set
up access to a host behind a bastion host.
Assuming, the host named host06
from above is behind a bastion host, a configuration
entry in $HOME/.ssh/config
like:
Host host06 User HOST06-USERNAME ForwardX11Trusted yes ProxyCommand ssh BASTION-USERNAME@ssh.example.com -W %h:%p
will allow the local account to use its SSH credentials for the user name
BASTION-USERNAME
on ssh.example.com
for access to host06
which
is on some protected network behind ssh.example.com
. When the local account
runs ssh host06
, SSH will first connect as BASTION-USERNAME
on
ssh.example.com
and then it connect to host host06
as username
HOST06-USERNAME
.
This sort of configuration will allow host06
to be allowed as a hostname
for remote kernels.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.