Tau

Tau is a Python runtime for distributed multi-projector rendering in the AlloSphere, with no dependency on allolib. State replicates over UDP, parameters sync over OSC, and each projection gets its own warp, blend, and quad-buffer stereo output.

Content can be an app class, functions registered on a runtime, a loop you own, or a plain object that the renderers draw. All four run in a window on one machine. The same code runs unchanged across the cluster.

In quantum physics, τ (tau) denotes the tangle, a measure of entanglement. For three or more parts it is what is left once every pairwise correlation has been accounted for: a property of the whole system, belonging to no pair within it.

Install

Use Tau from a clone. The examples, the tests, and the deploy scripts live in the repository tree.

git clone https://github.com/kr4g/Tau.git
cd Tau
python3 -m venv .venv
.venv/bin/python -m pip install -e .

In a tree of your own, install it as a dependency instead:

pip install tau-av

The package imports as tau.

python -m tau.preflight checks the machine it runs on: Python version, GL context, shader compilation, numba JIT, calibration, ports, UDP send and receive. Run it after installing, and on any machine before it joins a cluster.

Run

.venv/bin/python -m tau.launcher

The launcher lists the apps under examples/ and apps/ and runs the selection as a subprocess. When node agents run (see "Running in the AlloSphere"), a launch also switches what the cluster shows.

A single app runs directly:

.venv/bin/python -m examples.boids.app --view equirect

The views are pov (perspective, the default), cross (a cubemap box), equirect (a panorama), and anaglyph (a red/cyan stereo preview). The v key cycles them while the app runs. The arrow keys look around and WASD moves the camera. --help lists the other controls and flags.

A second instance on the same machine elects as a replica and follows the first. Every start prints a role banner: host, role, broadcast target, renderer, calibration.

Writing content

Apps you write go in apps/. Code they import goes in ext/. Both ship empty and git ignores their contents, so a pull never touches them. An app in apps/<name>/ runs as python -m apps.<name>.app, and the launcher discovers it there. The launcher lists an app when its app.py defines a top-level main(). An app.py without one still runs under python -m, but the launcher does not see it, and a cluster launch cannot start it.

An app class

Subclass DistributedApp and override its hooks. A numpy dtype declared as state_type becomes a replicated block: the primary writes it, and every renderer reads it.

class Cloud(tau.DistributedApp):
    state_type = np.dtype([("pos", np.float32, (256, 3))])
    sync_nav = True                          # replicas follow the primary camera

    def on_animate(self, dt):
        if self.is_primary():
            self.state()["pos"] += drift(dt)  # simulate on one machine

    def on_draw(self, g):
        ...                                   # draw on every machine

The other hooks are on_init, on_create (GL is live), on_gui(panel), and on_keys(keys). A registered Parameter syncs over OSC and appears in the control panel. Most examples follow one shape: a model module in vectorized numpy, a state.py with the dtype, and an app.py whose main() calls tau.run.

examples.collatz computes a large static layout once and loads it from the path in the TAU_CACHE_DIR environment variable, so every node reads one copy from a shared mount. This is a convention of the example, not of the runtime.

Registered functions

tau.runtime() builds a runtime whose hooks are registered functions:

rt = tau.runtime(state_type=my_dtype)

@rt.animate
def animate(dt):
    ...

@rt.draw
def draw(g):
    ...

rt.run()

A second registration replaces the first, even while the loop runs. The loop and the network connections do not stop when the content changes. With no draw hook the output is black, so a session can start empty and get content later. The swap is local to one process. Machines that did not run the registration keep the hooks they have. Changing what the cluster runs is a relaunch: the launcher stops the current module on every node and starts the new one.

Your own loop

rt.run() is a plain loop. A program with its own loop makes the same calls itself:

rt.open()             # election, domains, window
while running:
    rt.step(dt)       # simulate and replicate
    rt.poll(dt)       # domain upkeep
    rt.render()       # draw one frame
rt.shutdown()

This suits a notebook or a larger program that uses Tau as a library.

The Scene protocol

A renderer needs three things from the object it draws: a camera pose, a lens, and a draw callback. tau.Scene names this protocol, and any object that fills it in will do:

class Wrapped:
    def __init__(self, mesh):        # geometry from any source
        self._nav = tau.Nav()
        self._lens = tau.Lens()
        self._mesh = mesh

    def nav(self):
        return self._nav

    def lens(self):
        return self._lens

    def on_draw(self, g):
        g.clear(0.0)
        g.draw(self._mesh)

The class above subclasses nothing and registers nothing. The runtime takes it whole and renders it through the full capture, warp/blend, and stereo pipeline:

tau.runtime(scene=Wrapped(mesh)).run()

The runtime also calls optional hooks the object defines (on_animate, on_init, on_keys). Each machine calls on_draw on its own schedule, so content that keeps its own clock drifts across the cluster. Drive motion from replicated state or from the dt passed to on_animate.

This is the integration surface for content from other tools: wrap the content in the three methods, or give its per-frame arrays to the retained primitives (instanced meshes, points, lines, ribbons). State replication and parameters work alongside both.

The upload rule

on_animate(dt) runs once per frame. on_draw(g) runs once per projector per eye, so it must only issue draw calls. Build and upload geometry in on_animate. Content that uploads in on_draw looks correct in a window and renders differently on each projector. The checker below catches this.

Working with a coding agent

AGENTS.md carries the conventions and the reasons for them, written for an agent as much as for a person. An agent that works in a clone reads it from the repository root without being asked. The checker's --json output is for an authoring loop.

Stereo 3D

A calibrated renderer captures both eyes and presents them through a quad-buffer framebuffer. If the driver has no stereo framebuffer, the renderer warns and runs mono. --mono disables stereo.

The lens sets the depth. lens().focal_length(v) places the convergence distance. Content at that distance sits at the screen surface. Nearer content floats inside the sphere, and farther content recedes. lens().eye_sep(v) scales the disparity. Each example places its convergence where its content lives.

The /tau/stereo parameter is a checkbox in the control panel. It switches the cluster between stereo and mono while the app runs, and mono also halves the capture cost. A vertex shader gets the displacement when it calls stereo_displace(...). The runtime inserts the correct variant for the render path at compile time, and g.apply_stereo(prog) sets the uniforms.

At home, the anaglyph view shows the same disparity through red/cyan glasses. examples.calibration draws a depth ladder dead ahead. The graticule sits at the convergence distance. An orange ring at half that distance must float inside the sphere, and a violet ring at twice it must sit beyond. Flat rings mean stereo is dead. Swapped depths mean crossed eyes.

Control panel

On the simulator, every app gets a second window: a view selector, the stereo toggle, the pattern selector, the cluster roster, and a widget for each registered parameter. Overriding on_gui(panel) replaces the parameter widgets with custom UI. The other rows stay. Render nodes never open one, and --no-gui disables it.

Shaders

The renderer supplies tau_ModelViewMatrix, tau_ProjectionMatrix, and tau_ViewMatrix. A shader declares whichever it uses. Shaders loaded through ShaderManager reload on file change while the app runs.

Checking an app

.venv/bin/python -m tau.check apps.myapp    # or examples.boids, or a bare name

It runs the checks that work at home: shader compilation at the #version 410 ceiling, draw purity, uploads misplaced in on_draw, stereo, state size and wire rate, and headless determinism. The checker phrases each failure as the change that fixes it. --json emits the report for tooling. All it asks of the app is a smoke() function in app.py:

def smoke():
    return MyApp(n=64, seed=1, headless=True, fps=0.0)

smoke() can return a DistributedApp, a runtime built by tau.runtime (with hooks or a scene), or a bare Scene object. The checks that need replicated state skip when there is none.

The test suite (python -m pytest) checks the core the same way. It includes purity for every example, a pixel-for-pixel warp oracle, a two-process rehearsal of election and transport, and the stereo gates. bash scripts/test-py310.sh repeats the suite on the renderers' interpreter. python -m tests.bench_sphere reports timing costs, and --save / --compare bracket a change. Subnet broadcast, driver differences, the spanned X screen, and quad-buffer presentation are checked on site.

No check covers whether content reads from inside the AlloSphere. Content can pass everything and still be composed for a rectangle. Look at it in --view equirect and --view pov.

Running in the AlloSphere

Stage the work on the shared /alloshare mount. Build the venv there once, from a renderer: bash deploy/build_venv.sh. Then start the same app on every machine by hand:

.venv/bin/python -m examples.<name>.app     # identical on every machine

or start the node agents once and switch content from the simulator:

bash deploy/launch_sphere.sh start
.venv/bin/python -m tau.launcher

Role and renderer resolve from the hostname and the calibration data. The primary host (ar01, or whatever TAU_PRIMARY_HOST names) simulates and sends state. A renderer with a calibration manifest applies warp, blend, and stereo, and runs fullscreen. --sim makes any machine the primary.

State ships as full snapshots over UDP, latest-wins. A change to the state dtype needs a relaunch on every node: a node running a different dtype drops the mismatched snapshots and shows NO STATE on its output.

Every node sends a heartbeat once a second. The control panel shows the roster, and python -m tau.heartbeat prints the same table. Two simultaneous primaries appear on both.

The /tau/pattern parameter (control panel, or keys 0–5 in examples.calibration) switches every renderer into a projector-identification pattern without stopping the content. 0 returns to normal. Degraded states, such as missing calibration or state that stops arriving, appear on the output itself.