async-kernel 0.13.2

A concurrent python kernel for Jupyter supporting AnyIO, AsyncIO and Trio.

async-kernel

async-kernel is a Python kernel for Jupyter that provides concurrent message handling via an asynchronous backend (asyncio or trio).

The kernel provides two external interfaces:

1. Direct ZMQ socket messaging via a configuration file and kernel spec - (Jupyter, VScode, etc).
2. An experimental callback style interface (Jupyterlite).

Highlights

• IPython shell
• anyio compatible asynchronous backend (asyncio (default) or trio)
• aiologic thread-safe synchronisation primitives
• Backend agnostic multi-thread / multi-event loop management
• Per-subshell user_ns
• GUI event loops [^1]
  • inline
  • ipympl
  • tk host and asyncio[^2] or trio[^3] backend running as a guest
  • qt host and asyncio[^2] or trio[^3] backend running as a guest
• Experimental support for Jupyterlite (try it online here 👈)
• Debugger client

[^1]: A gui (host) enabled kernel runs a gui event loop with the asynchronous backend running as guest. The host must be set before the kernel is started. This was a deliberate design choice to to ensure good performance and reliability.

[^2]: The asyncio implementation of start_guest_run was written by the author of aiologic and provided as a gist.

[^3]: trio's start_guest_run.

Avoid deadlocks

The standard (synchronous) kernel implementation processes messages sequentially irrespective of the message type. The problem being that long running execute requests make the kernel non-responsive.

Another problem exists when an asynchronous execute request awaits a result that is delivered via a kernel message - this will cause a deadlock because the message will be stuck in the queue behind the blocking execute request[^5].

async-kernel handles messages according to the channel, message type and subshell id. So widget com message will get processed in a separate queue to an execute request. Further detail is given in the concurrency notebook, a Jupyterlite version is available here.

Example

Try the following using a standard kernel and then try it with async-kernel.

# Make the shell thread busy
import time

time.sleep(1e6)

Try the following in another cell:

• code completion (tab)
• docstring (shift tab)

[^5]: IPyKernel solves this issue specifically for widgets by using the concept of 'widget coms over subshells'. Widget messages arrive in a different thread which on occasion can cause unexpected behaviour, especially when using asynchronous libraries.

Installation

pip install async-kernel

Kernel specs

A kernel spec with the name 'async' is added when async-kernel is installed.

Kernel specs can be added/removed via the command line.

Backends

The backend set on the interface is the asynchronous library the kernel uses for message handling. It is also the asynchronous library directly available when executing code in cells or via a console[^3].

[^3]: Irrespective of the configured backend, functions/coroutines can be executed using a specific backend with the method call_using_backend.

Example - overwrite the 'async' kernel spec to use a trio backend

pip install trio
async-kernel -a async --interface.backend=trio

Gui event loop

The kernel can be started with a gui event loop as the host and the backend running as a guest.

asyncio backend

# tk
async-kernel -a async-tk --interface.loop=tk

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt

trio backend

pip install trio
# tk
async-kernel -a async-tk --interface.loop=tk --interface.backend=trio

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt --interface.backend=trio

For further detail about kernel spec customisation see command line and kernel configuration.

Origin

async-kernel started as a fork of IPyKernel. Thank you to the original contributors of IPyKernel that made async-kernel possible.