jupyter-shelf 0.0.4

A CLI tool to manage virtual environments for jupyter notebooks

Jupyter Shelf

If you are using Jupyter notebooks extensively and need a convenient way to use virtual environments with the notebooks, Shelf may be suitable for you.

Shelf allows you create a virtual environment and use it with a set of notebooks in a Jupyter lab environment, hence the name shelf.

It is easy to create and dispose shelves. Shelf can help you organize your workspace and notebooks.

Why Shelf was needed ?

I use Jupyter notebooks for a wide range of tasks ranging from testing python libraries&tools to training&testing language models. Most of the time testing requires a different python environment. I always needed to use Jupyter notebooks with use virtual environments. You need to run several commands to create a kernel from a virtual environment and then you can use the kernel within the jupyter lab. This manual approach kept me away from using virtual environments. Then I developed several bash functions to alleviate the burden of managing virtual environment kernels. Then I realized that I need a real solution that I can use in different environments.

Instructions

Installation

pip install jupyter-shelf --upgrade

Print Help:

shelf -h

Sample output:

usage: shelf [-h] {mk,rm,ls,start,stop} ...

positional arguments:
  {mk,rm,ls,start,stop}
    mk                  Create a new shelf.
    rm                  Remove specified shelf. Retains notebooks.
    ls                  List shelves in the given root directory.
    start               Start jupyter lab in the specified shelf.
    stop                Stop jupyter lab running in the specified shelf.

options:
  -h, --help            show this help message and exit

Trouble shooting

It is recommended to install and use shelf command in a virtual environment.

Outside of the virtual environment it is possible to use the shelf command but you may experience issues due to your system configuration or pip behavior. If you install outside a virtual environment and shelf command does not work, then your will need to find the shelf script and add it to path variable.

pip3 show jupyter_shelf

In may case it was installed under /home/<user.home>/.local/lib/python3.10/site-packages directory. Then shelf script is found under /home/<user.home>/.local/bin/ directory. Adding this directory to path variable should fix the problem. Edit your .bashrc file to add following line.

export PATH=$PATH:/home/<user.home>/.local/bin/

How to use?

Make a shelf

Create a shelf named hf-demo under the given root directory.

shelf mk hf-demo --root ~/workspace/shelves --python-version 3.11

This will create ~/workspace/shelves/hf-demo directory. Initialize a virtual environment there. Install jupyter-lab. And create src directory to keep notebooks. It will also create a virtual environment kernel that shares the same name with the shelf. The kernel is only accessible to jupyter process running in this shelf. Once the jupyter windows is open, look for the kernel named after your shelf name. Select it and start using the virtual environment of the shelf.

~/workspace/shelves is the default shelf root and can be omitted. A typical usage would look like this:

shelf mk torch-demo

List all shelves under a root directory.

shelf ls --root ~/workspace/shelves

Again ~/workspace/shelves is the default root and can be omitted.

Start jupyter lab

Following command will start jupyter lab in the given shelf.

shelf start hf-demo --root ~/workspace/shelves --port 8888

Jupyter-lab will start in src sub-directory of the shelf. 8888 is the default port used by jupyter and can be omitted. If collusion occurs, jupyter will try next port. So typical usage would look like this:

shelf start hf-demo

Send CTRL+C to kill the Jupyter process. You can have multiple jupyter processes starting from different shells but each one will block its shell until you send CTRL+C. Starting multiple jupyter processes in the same shelf is possible but not recommended.

Remove a shelf

Following command will remove a shelf directory.

shelf rm hf-demo --root ~/workspace/shelves

This command will not remove src directory if it contains any files. This is to ensure your source files are not wasted. This command does not have an override option. If you are sure about deleting your source files, you will need to do it manually.

Commands to publish

Make sure you are up-to-date.

python3 -m pip install --upgrade pip
python3 -m pip install --upgrade build
python3 -m pip install --upgrade twine

Build and publish

python3 -m build
python3 -m twine upload --repository pypi dist/*

References

1. Packaging
2. Entry point
3. Sample Setup-tools Project