viser4d 0.7.2

A viser extension with out-of-the-box support for the time dimension

viser4d

viser4d is a small wrapper around viser that adds a time dimension. It records scene operations across timesteps, supports timeline-synced audio playback, and plays them back client-locally in each browser tab.

Quickstart

pip install viser4d

import numpy as np
import viser4d

server = viser4d.Viser4dServer(num_steps=10, fps=10)

with server.at(0):
    points = np.random.uniform(-1.0, 1.0, size=(200, 3))
    point_cloud = server.scene.add_point_cloud(
        "/points",
        points=points,
        colors=(255, 200, 0),
    )

for i in range(1, 10):
    with server.at(i):
        points = np.random.uniform(-1.0, 1.0, size=(200, 3))
        point_cloud.points = points

server.sleep_forever()

Open the viewer in your browser and use the built-in Playback controls to play, scrub, and step through the client-local timeline.

Timeline model

• The built-in browser controls (Play, Pause, Prev, Next, and the Timestep slider) are client-local. Different tabs can be on different timesteps at the same time, and those controls are handled directly in the browser rather than round-tripping through Python.
• The fps= passed to Viser4dServer(...) defines the timeline step rate used for audio timing and .viser export, and also serves as the initial client playback speed.
• server.on_timestep_change(...) fires whenever any client commits a new discrete timestep and passes (client, timestep). With multiple clients, it is an aggregate event stream and may repeat timesteps or arrive out of order.
• server.on_playback_change(...) fires whenever a client reports that its built-in transport changed between playing and paused, and passes (client, is_playing).
• server.play(...) and server.pause() broadcast playback commands to the clients that are connected right now. They do not create a shared server clock.

Streaming ingest

If data arrives incrementally, initialize components at t=0 and then record updates as each new frame arrives:

import numpy as np
import viser4d

num_steps = 180
server = viser4d.Viser4dServer(num_steps=num_steps, fps=30)

def get_next_points() -> np.ndarray:
    # Replace with your real sensor/network/pipeline frame source.
    return np.random.normal(size=(400, 3)).astype(np.float32)

with server.at(0):
    point_cloud = server.scene.add_point_cloud(
        "/stream/points",
        points=get_next_points(),
    )

for t in range(1, num_steps):
    points = get_next_points()
    with server.at(t):
        point_cloud.points = points

server.sleep_forever()

Timestep callbacks

If you have your own visualization logic and just want to use viser4d's timeline infrastructure, you can register a callback that fires whenever any connected client commits a new discrete timestep:

import viser
import viser4d

server = viser4d.Viser4dServer(num_steps=100)

def on_timestep(client: viser.ClientHandle, t: int) -> None:
    update_video_frame(client.scene, t)
    update_client_overlays(client.scene, t)

server.on_timestep_change(on_timestep)
server.sleep_forever()

With multiple clients, this callback is aggregate: if two tabs both visit timestep 3, it will fire twice, once for each client.

Playback state callbacks

If you need to know when a client starts or stops playback, use the playback callback and the per-client playback handles:

import viser
import viser4d

server = viser4d.Viser4dServer(num_steps=100)

def on_playback_change(client: viser.ClientHandle, is_playing: bool) -> None:
    print(client.client_id, is_playing)

server.on_playback_change(on_playback_change)

# Snapshot of connected playback handles keyed by client id.
for client_id, playback in server.get_client_playbacks().items():
    print(client_id, playback.is_playing, playback.current_timestep)

ClientPlaybackHandle.is_playing reflects the last play/pause state reported by that browser tab. server.play(...) and server.pause() send commands, but the handle state only changes once the client reports the result back.

Server playback commands

server.play(...) starts each connected client from that client's own current timestep. Passing fps=... to server.play(...) also updates the default client playback speed for later play() calls and future clients. server.pause() pauses each connected client wherever it currently is. server.set_fps(...) updates the same default playback speed without starting playback. Neither method changes the timeline step rate used for audio timing or export; set that with fps= when you construct the server. New clients always start paused at timestep 0.

Serialize .viser recordings

To serialize the full viser4d timeline, including audio, use server.serialize():

import viser4d

server = viser4d.Viser4dServer(num_steps=100)
# ... record timeline data ...
blob = server.serialize(start_timestep=0, end_timestep=None)

Write the returned bytes to disk yourself if needed.

Streaming audio append

For audio that arrives incrementally, create a track once inside at(t) and append chunks through the returned handle:

import numpy as np
import viser4d

server = viser4d.Viser4dServer(num_steps=300, fps=30)

with server.at(0):
    audio = server.audio.add_track(
        "/stream/audio",
        data=np.zeros(1600, dtype=np.float32),
        sample_rate=16000,
    )

for _ in range(120):
    chunk = np.random.uniform(-0.05, 0.05, size=(1600,)).astype(np.float32)
    audio.append(chunk)

AudioHandle.append(...) extends the same track contiguously (same channel count).

How it works

Context determines behavior. server.scene is viser's normal scene API, but its websocket target is swapped while you're inside an at(t) context:

Inside at(t):                          Outside at(t):
─────────────                          ──────────────
scene.add_frame(...)                   scene.add_frame(...)
       │                                      │
       ▼                                      ▼
    records to Timeline                    forwards to live viser scene

• Inside at(t): Operations are recorded to a timeline, not executed.
• Outside at(t): Operations forward directly to viser's live scene.
• Client playback: Each browser tab owns its own transport and playback state.
• Timestep callbacks: on_timestep_change(...) aggregates committed client steps and passes the source client.
• Playback callbacks: on_playback_change(...) reports per-client play/pause transitions.
• Audio: Add timeline-synced tracks with server.audio.add_track(...).

See examples/ for more.

Quality checks

uvx ruff format .
uvx ruff check .
uvx ty check
pnpm run typecheck:runtime
pnpm run build:runtime

Tests

pnpm run build:runtime
uv run --group dev pytest -q