async-kernel 0.13.0

A concurrent python kernel for Jupyter supporting AnyIO, AsyncIO and Trio.

async kernel

async kernel is a Python kernel for Jupyter that provides concurrent message handling via an asynchronous backend (asyncio or trio).

The kernel provides two external interfaces:

1. Direct ZMQ socket messaging via a configuration file and kernel spec - (Jupyter, VScode, etc).
2. An experimental callback style interface (Jupyterlite).

Highlights

• IPython shell
• anyio compatible asynchronous backend (asyncio (default) or trio)
• aiologic thread-safe synchronisation primitives
• Backend agnostic multi-thread / multi-event loop management
• Per-subshell user_ns
• GUI event loops [^1][^2]
  • inline
  • ipympl
  • tk host and asyncio[^3] or trio[^4] backend running as a guest
  • qt host and asyncio[^3] or trio[^4] backend running as a guest
• Experimental support for Jupyterlite (try it online here 👈)
• Debugger client

[^1]: A gui (host) enabled kernel runs a gui event loop with the asynchronous backend running as guest. The host must be set before the kernel is started. This was a deliberate design choice to to ensure good performance and reliability.

[^2]: It is also possible to use a caller to run a gui event loop in a separate thread (with a backend running as a guest) if the gui allows it (qt will only run in the main thread). Also note that pyplot will only permit one interactive gui library per process.

[^3]: The asyncio implementation of start_guest_run was written by the author of aiologic and provided as a gist.

[^4]: trio's start_guest_run.

Prevent asynchronous deadlocks

The standard (synchronous) kernel implementation processes messages sequentially irrespective of the message type. The problem being that long running execute requests will make the kernel non-responsive. Another problem exists when an asynchronous execute request awaits a result that is delivered via a kernel message - this will cause a deadlock because the message will be stuck in the queue behind the blocking execute request[^5].

async kernel handles messages according to the channel, message type and subshell id. So widget com message will get processed in a separate queue to an execute request. Further detail is given in the concurrency notebook, a Jupyterlite version is available here.

[^5]: IPyKernel solves this issue specifically for widgets by using the concept of 'widget coms over subshells'. Widget messages arrive in a different thread which on occasion can cause unexpected behaviour, especially when using asynchronous libraries.

Installation

pip install async-kernel

Kernel specs

A kernel spec with the name 'async' is added when async kernel is installed.

Kernel specs can be added/removed via the command line.

Backends

The backend set on the interface is the asynchronous library the kernel uses for message handling. It is also the asynchronous library directly available when executing code in cells or via a console[^4].

[^4]: Irrespective of the configured backend, functions/coroutines can be executed using a specific backend with the method call_using_backend.

Example - overwrite the 'async' kernel spec to use a trio backend

pip install trio
async-kernel -a async --interface.backend=trio

Gui event loop

The kernel can be started with a gui event loop as the host and the backend running as a guest.

asyncio backend

# tk
async-kernel -a async-tk --interface.loop=tk

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt

trio backend

pip install trio
# tk
async-kernel -a async-tk --interface.loop=tk --interface.backend=trio

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt --interface.backend=trio

For further detail about kernel spec customisation see command line and kernel configuration.

Origin

async kernel started as a fork of IPyKernel. Thank you to the original contributors of IPyKernel that made async kernel possible.