async-kernel 0.20.0

A concurrent python kernel for Jupyter supporting AnyIO, AsyncIO and Trio.

async-kernel

async-kernel is a Python kernel for Jupyter that provides concurrent message handling via an asynchronous backend (asyncio or trio).

The kernel provides two external interfaces:

1. Direct ZMQ socket messaging via a configuration file and kernel spec - (Jupyter, VS Code, etc).
2. An experimental callback style interface (Jupyterlite).

Highlights

• IPython shell
  • top-level await ('asyncio' or 'trio' backend) in cells
  • async magic function support in cells
• anyio compatible asynchronous backend (asyncio (default) or trio)
• aiologic thread-safe synchronisation primitives
• Backend agnostic multi-thread / multi-event loop management
• Per-subshell user_ns
• GUI event loops ¹
  • inline
  • ipympl
  • tk host and asyncio² or trio³ backend running as a guest
  • qt host and asyncio² or trio³ backend running as a guest
• Experimental support for Jupyterlite (try it online here 👈)
  • %pip install magic (using micropip)
• Debugger client

Avoid deadlocks

The standard (synchronous) kernel implementation processes messages sequentially irrespective of the message type. The problem being that long running execute requests make the kernel non-responsive.

Another problem exists when an asynchronous execute request awaits a result that is delivered via a kernel message - this will cause a deadlock because the message will be stuck in the queue behind the blocking execute request⁴.

async-kernel handles messages according to the channel and message type. So widget com message will get processed in a separate queue to an execute request. Further detail is given in the concurrency notebook, a Jupyterlite version is available here.

Example

Make a blocking call in a Jupyter lab notebook or console.

# Make the shell's thread busy
import time

time.sleep(1e6)

While the above is blocking (the kernel is busy).

dir()  # try code completion (tab) or view the docstring (shift tab)

Interrupt the kernel.

It also works for awaitables.

import ipywidgets as ipw
from aiologic import Event

b = ipw.Button(description="Click me")
event = Event()
b.on_click(lambda _: event.set())
display(b)
await event

Installation

pip install async-kernel

Kernelspecs

A kernelspec with the name 'async' is added when async-kernel is installed.

Kernel specs can be installed/uninstalled via the command line.

async-kernel install

# To install for a user
async-kernel install --user

For further detail about kernel spec customisation see command line and kernel configuration and custom kernel.ipynb.

Faster data serialization

orjson (a fast JSON library) is supported and will be used by default if it has been installed.

Free-threading support

async-kernel's Caller's are thread-local and it's methods are internally synchronised⁵.

Origin

async-kernel started as a fork of IPyKernel. Thank you to the original contributors of IPyKernel that made async-kernel possible.

1. A gui (host) enabled kernel interface starts a gui's mainloop (host) which starts the backend as a guest, then finally the Kernel is started. ↩

2. The asyncio implementation of start_guest_run was written by the author of aiologic and provided as a gist. ↩ ↩²

3. trio's start_guest_run. ↩ ↩²

4. IPyKernel solves this issue specifically for widgets by using the concept of 'widget coms over subshells'. Widget messages arrive in a different thread which on occasion can cause unexpected behaviour, especially when using asynchronous libraries. ↩

5. free threading terminology ↩