browseraudio

Audio I/O for in-browser Python (Pyodide, JupyterLite, thebe): record, play, and synthesize in real time, to and from NumPy.

Browser kernels usually run in a Web Worker, which can't reach the Web Audio API. browseraudio does the audio on the page's main thread through small anywidget frontends and moves the samples over the widget comm.

Live demo →

Install

pip install browseraudio

In a browser kernel: import micropip; await micropip.install("browseraudio"). Needs a widget-capable kernel (JupyterLite, thebe, Jupyter, marimo) and, for the microphone, a secure context (https:// or localhost).

Quickstart

from browseraudio import record, play, keyboard, stream

rec = record(3.0)                 # click Record; read rec.samples in the NEXT cell
play(rec.samples, rec.sample_rate)

Recording finishes after the cell returns (a Web-Worker kernel can't receive the take mid-cell), so read rec.samples in a later cell.

Keyboard: design a sound, then play it on the computer keyboard (A W S E D …, Z/X for octaves). Notes are voiced in the page, polyphonic, with no kernel round trip:

import numpy as np
sr = 22050
t = np.arange(sr) / sr
kb = keyboard(lambda f: np.sin(2 * np.pi * f * t) * np.exp(-4 * t), sr)

Stream: real-time synthesis. The browser pulls blocks from your callback:

class Sine:
    def __init__(self):
        self.pitch, self.phase, self.on, self.amp = 60, 0.0, False, 0.0

    def callback(self, block):                     # (B, 1) with .sample_rate
        B = len(block)
        f = 440 * 2 ** ((self.pitch - 69) / 12)
        target = 0.3 if self.on else 0.0
        amp = np.linspace(self.amp, target, B, endpoint=False)  # glide: no clicks
        d = 2 * np.pi * f / block.sample_rate
        block[:, 0] = amp * np.sin(self.phase + d * np.arange(B))
        self.phase, self.amp = (self.phase + d * B) % (2 * np.pi), target

synth = Sine()
def on_note(midi, on):
    if on: synth.pitch, synth.on = midi, True
    elif midi == synth.pitch: synth.on = False

s = stream(synth.callback, sample_rate=44100, block_size=512, on_note=on_note)

The widget shows block size, queue, round trip, callback load, dropouts and key-to-sound latency. For chords, keep one voice per held key (see the demo).

Genie mode: pass input="genie" to keyboard or stream, or use the widget's Keys | Genie switch. Eight buttons (1–8 or A S D F J K L ;) become piano music via Piano Genie.

sounddevice code runs unchanged once the facade is registered:

import sys, browseraudio.sounddevice
sys.modules["sounddevice"] = browseraudio.sounddevice

API

How it works

Widgets render in a Shadow DOM. They follow the JupyterLab theme, or the host page's text colour and font elsewhere. Audio crosses the comm as base64 float32. For streams the browser is the clock: an AudioWorklet plays a short queue and asks the kernel for each next block. A kernel answers whenever no cell is running, even in a Web Worker, so a control change is heard after the queue plus one round trip (about 50–100 ms). Dropouts and Stop fade out rather than click.

Troubleshooting

Development

pip install -e ".[test]" && pytest     # genie tests also need node
python -m build && python -m twine check dist/*

Python is in browseraudio/, frontends in browseraudio/static/ (ui.js is shared; genie.js and controls.js are added for keyboard widgets), and the demo site in demo/.

Credits

Genie mode is Piano Genie. It runs Magenta's published model (Apache-2.0) with inference ported from Magenta.js (Apache-2.0). Please cite:

@inproceedings{donahue2019pianogenie,
  title     = {Piano Genie},
  author    = {Donahue, Chris and Simon, Ian and Dieleman, Sander},
  booktitle = {Proceedings of the 24th International Conference on Intelligent User Interfaces},
  pages     = {160--164},
  year      = {2019},
  doi       = {10.1145/3301275.3302288}
}

License

MIT