Skip to main content

A Jupyter kernel using LLM models from Huggingface

Project description

chatbot_kernel

A Jupyter kernel to use Jupyter as a chat window, running downloaded LLMs locally.

Installation

This kernel package can be install from PyPI

$ pip install chatbot_kernel

Once the package installed, the kernel spec can be installed to home directory by command:

$ python -m chatbot_kernel install --user

If you are using virtualenv, do

$ python -m chatbot_kernel install --sys-prefix

instead.

If you install the package to a virtual environment, you may need to set up JUPYTER_PATH=/path/to/venv/share/jupyter:$JUPYTER_PATH so that jupyter can find the kernel

Usage

A few magics are available in the kernel:

  • %help: Print help messages
  • %config: Set advanced configuration. See more in %config help
  • %load: Load a pretrained LLM
  • %hf_home: Set the path to find downloaded LLMs, similar to set HF_HOME environment variable
  • %model_list: Show the available LLMs
  • %new_chat: Clean up the chat history

Before start chatting, you need to at least download a model from HuggingFace. For example, huggingface-cli download meta-llama/Meta-Llama-3-8B-Instruct. Once models are downloaded, launch a jupyter notebook/lab and execute %load <model> and start chatting. Here is an example:

%load meta-llama/Meta-Llama-3-8B-Instruct
hi 
who are you

Caveat

Currently, the kernel use the AutoModelForCausalLM and it is not supported by all models. A few models have been tested:

  • meta-llama/Meta-Llama-3-8B-Instruct
  • mistralai/Mistral-7B-Instruct-v0.3: needs sentencepiece dependency
  • unsloth/llama-3-8b-Instruct-bnb-4bit: needs bitsandbytes dependency

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

chatbot_kernel-0.3.3.tar.gz (8.8 kB view details)

Uploaded Source

Built Distribution

chatbot_kernel-0.3.3-py3-none-any.whl (9.4 kB view details)

Uploaded Python 3

File details

Details for the file chatbot_kernel-0.3.3.tar.gz.

File metadata

  • Download URL: chatbot_kernel-0.3.3.tar.gz
  • Upload date:
  • Size: 8.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.27.0

File hashes

Hashes for chatbot_kernel-0.3.3.tar.gz
Algorithm Hash digest
SHA256 b498dcf8bf053716b1e3d3d91832c9db15f011424d97fd3678c43021b8e1d3fd
MD5 32dd2872ab3bb2cdebbe971cf39d1fca
BLAKE2b-256 da188e79bd566c68ab0f79d0fb0bf0e1f22004891c91befa43b350e99fbfbd51

See more details on using hashes here.

File details

Details for the file chatbot_kernel-0.3.3-py3-none-any.whl.

File metadata

File hashes

Hashes for chatbot_kernel-0.3.3-py3-none-any.whl
Algorithm Hash digest
SHA256 6f7e26dcf9acfaa3b61ed68ea12eeb7984211f31268a6d55ba345fbb7571399b
MD5 6d3ca017677b482e4a589e65c67487bd
BLAKE2b-256 8b44e93dad51183b2f0e7eb600123bd8b0a462a2c0a7a5a2f79598c483374744

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page