lux

A visual output surface for AI agents.

Lux gives agents and apps a shared visual surface. The intended architecture is a hub/display split: clients send UI descriptions to luxd, the Hub owns authoritative element state and behavior, and the Display renders a replica of the current scene while forwarding user interactions back to the Hub.

The design draws on X11's client/server split and Smalltalk-style live introspection. MCP is one gateway into Lux, not the whole architecture. If you want the short version of the rewrite target, start with docs/architecture/target/target.md. If you need help navigating the docs, use docs/README.md. For the product direction, positioning, and risk assessment — the Working Backwards PR/FAQ — see prfaq.pdf.

Platforms: macOS, Linux

Stage: alpha --- protocol is stable, published on PyPI as punt-lux

A Claude Code plugin displaying a project issue board --- the agent fetches live data from DoltDB via bd list --json, builds a filterable table with detail panel, and renders it in a single tool call. Filters and row selection run at 60fps with zero MCP round-trips.

The same list/detail pattern generalizes to any tabular data. Search, combo filters, pagination, and a detail panel --- all driven by a single show_table() call.

Dashboards compose metric cards, charts, and tables. show_dashboard() builds the layout from structured data --- no manual element positioning needed.

Quick Start

curl -fsSL https://raw.githubusercontent.com/punt-labs/lux/1117250/install.sh | sh

Restart Claude Code twice. The Lux display window opens automatically when agents send visual output.

uv tool install 'punt-lux[display]'

claude plugin marketplace add punt-labs/claude-plugins
claude plugin install lux@punt-labs

uv add punt-lux

curl -fsSL https://raw.githubusercontent.com/punt-labs/lux/1117250/install.sh -o install.sh
shasum -a 256 install.sh
cat install.sh
sh install.sh

lux display &
uv run python demos/dashboard.py

Features

• 25 element kinds --- text, buttons (arrow, small), images, sliders, checkboxes, combos, inputs (text, number), radios, color pickers (alpha, full picker), selectables, trees, tables, plots, progress bars, spinners, markdown, draw canvases, modals, dialogs, groups, tab bars, collapsing headers, windows, separators
• Frames --- scenes target named frames (inner windows) via frame_id. Frames persist after disconnect, can be adopted by new clients, and support initial sizing (frame_size) and ImGui window flags (frame_flags)
• Layout nesting --- windows contain tab bars contain groups contain any element, arbitrarily deep
• Incremental updates --- update patches individual elements by ID without replacing the scene
• Session menus --- the menu bar shows one submenu per live session. A session registers a menu entry via register_callback; a click on it is held for the owning session, which services it from its own shell. The built-in "Beads" entry each lux-enabled session registers is how the beads board reopens from the menu
• Interaction handling --- button clicks, slider changes, and menu clicks fire their handlers on the Hub (D21 remote dispatch); the raw event log is readable via list_recent_events. Hub handlers can publish app events that the agent reads via recv
• Frame auto-focus --- frames automatically focus (brought to front) when they receive a scene update
• Persistent tabs --- each show() call opens a dismissable tab; same scene_id replaces content in-place. Users can close individual tabs
• Themes --- 11 themes via set_theme: imgui_colors_dark, imgui_colors_light, imgui_colors_classic, darcula, darcula_darker, material_flat, photoshop_style, grey_flat, cherry, light_rounded, microsoft_style
• Auto-spawn --- the Hub (luxd) starts the display renderer on first use if it isn't already running
• Unix socket IPC --- length-prefixed JSON frames, no HTTP overhead, no threads

MCP Tools

Agents interact with Lux through 30 MCP tools that luxd serves over its streamable-HTTP /mcp endpoint:

What It Looks Like

Show text and a button

{"tool": "show", "input": {
  "scene_id": "hello",
  "elements": [
    {"kind": "text", "id": "t1", "content": "Hello from the agent"},
    {"kind": "button", "id": "b1", "label": "Click me"}
  ]
}}

Returns "shown:hello" immediately — the Hub has accepted the scene and its background replicator paints it; no tool call ever waits on the display. A button click fires its handler on the Hub (the agent does not poll for it). To observe interactions, read the introspection log:

{"tool": "list_recent_events", "input": {"count": 5}}

A Hub-side handler can publish an app event that the agent then reads with recv (see the Pub/Sub tools above).

Multi-window dashboard

{"tool": "show", "input": {
  "scene_id": "dash",
  "elements": [
    {"kind": "window", "id": "w1", "title": "Controls", "x": 10, "y": 10,
     "children": [
       {"kind": "slider", "id": "vol", "label": "Volume", "value": 50}
     ]},
    {"kind": "window", "id": "w2", "title": "Chart", "x": 320, "y": 10,
     "children": [
       {"kind": "plot", "id": "p1", "title": "Trend",
        "series": [{"label": "y", "type": "line",
          "x": [1,2,3,4], "y": [10,20,15,25]}]}
     ]}
  ]
}}

Update a single element

{"tool": "update", "input": {
  "scene_id": "dash",
  "patches": [
    {"id": "vol", "set": {"value": 75}}
  ]
}}

Element Kinds

All elements with an id support an optional tooltip field (string shown on hover).

CLI Commands

Library (Python)

LuxRestClient is the public Python client of luxd. A downstream application reaches the Hub through this typed client — not by hand-rolling REST calls — so it gets the same validation, typing, and identity handling the CLI does. It imports without the [display] extra (no ImGui/OpenGL pulled in).

from punt_lux import LuxRestClient, RenderRequest, SceneShown

# connect() locates a running luxd and derives this invocation's identity from
# the git repository it runs in — so the scene below is owned by that repo.
client = LuxRestClient.connect()

result = client.render(
    RenderRequest(
        scene_id="hello",
        elements=[{"kind": "text", "id": "t1", "content": "Hello from Python"}],
    )
)
if isinstance(result, SceneShown):
    print("shown:", result.scene_id)

Every request carries the caller's X-Lux-Client-* identity headers, so each installed scene is attributed to its repository rather than an anonymous caller. An unreachable luxd raises HubUnavailableError; a reachable Hub's refusal of a request comes back as a typed OpError in the result.

Listening: a persistent hub client

A daemon that wants to receive — pub-sub events it subscribed to, and the menu callbacks the user clicked — holds one WebSocket to luxd with LuxHubClient. It shares the identity of a LuxRestClient, so a callback the daemon registers over REST is delivered on this stream. The receive loop renews the lease on every contact and reconnects on a dropped connection, re-subscribing automatically; the Hub buffers any clicks missed during a gap and drains them on reconnect.

Re-subscribing restores topics, but a menu callback lives on the session's lease, which lapses during a long outage — so register it (and re-push scenes) in on_connect, which runs after every handshake, first connect and each reconnect. Registering in an outer register-then-listen sequence would run once; the internal reconnect would never re-run it, and the menu entry would stay gone.

import asyncio

from punt_lux import ClientIdentity, LuxRestClient

# One identity for both legs: scene pushes over REST, the listen stream over the
# WebSocket. A long-lived daemon declares an "app" identity — who it is, not where
# it ran — and a short lease TTL so its menu entries leave when it dies.
rest = LuxRestClient.for_identity(ClientIdentity(kind="app", name="voxd", lease_ttl=30))

def on_callback(callback_id: str) -> None:
    print("menu click:", callback_id)          # e.g. run the action for this item

def on_event(topic: str, payload: dict[str, object]) -> None:
    print("event:", topic, payload)            # e.g. {"album_id": "jazz-1"}

def on_connect() -> None:
    # Runs after every handshake — re-establish the per-connection state the
    # reconnect does not: register menu callbacks, re-push any scenes.
    rest.register_callback("music", "Music")

listener = rest.listener(
    on_callback=on_callback, on_event=on_event, on_connect=on_connect
)
listener.subscribe("music.play", "music.stop")
asyncio.run(listener.listen())                 # blocks, reconnecting as needed

The handlers may be sync or async; the loop awaits a coroutine. A raising on_connect is logged and the connection continues — a failed re-registration never tears down a healthy socket. Call stop() to end the loop after its current connection closes. Events and callbacks are generic — the topics and callback ids are the app's own vocabulary, not Lux's.

Architecture

Agent or app
  │ MCP or direct Hub API
  ▼
luxd (Hub)
  │ authoritative state + introspection
  │ scene replicas + remote invocations
  ▼
lux display (ImGui + OpenGL)
  │ renders at 60fps
  ▼
Window on screen

The Hub is the single source of truth for element state, ownership, and handler dispatch. The Display is a rendering replica: it paints the current scene and forwards interactions back to the Hub, which runs the real handler and re-pushes updated state. MCP is one entry point, not the only one.

Connecting Claude Code directly over HTTP

luxd serves MCP over streamable HTTP at http://127.0.0.1:8430/mcp, on the same loopback port as its REST API. Claude Code can connect to that endpoint natively through its HTTP MCP configuration, with no mcp-proxy bridge in the path. Point Claude Code's MCP config (a project .mcp.json or the plugin's mcpServers block) at the endpoint:

{
  "mcpServers": {
    "lux": { "type": "http", "url": "http://127.0.0.1:8430/mcp" }
  }
}

The bundled plugin ships exactly this HTTP config in its mcpServers block — no mcp-proxy bridge and no lux serve stdio fallback. The installer registers luxd as a launchd service pinned to --port 8430, so the static URL is correct on installed systems; if you run luxd on a non-default port, read the real one from the port file (~/.punt-labs/lux/hub.port, i.e. HubPaths().read_port()) and set the URL to match.

A copy-paste example is in .claude-plugin/mcp-http.example.json. Start luxd first (lux hub-install and start the service, or run luxd in a terminal), then verify the direct connection end to end:

uv run python scripts/direct_connection_probe.py

The probe initializes a session, lists the tool surface, and calls a read-only tool. luxd binds loopback only and refuses a non-loopback --host at startup; remote access awaits authentication.

Documentation

Docs Guide | Target Architecture | Target Topology | Target UI Model | Target Introspection | Current Architecture | Design Log | Changelog

Development

uv sync --extra display       # Install dependencies (dev group installs by default)
uv run ruff check .            # Lint
uv run ruff format --check .   # Check formatting
uv run mypy src/ tests/        # Type check (mypy)
uv run pyright                 # Type check (pyright)
uv run pytest                  # Test

Acknowledgements

Lux is a thin orchestration layer. The rendering is done by Dear ImGui, Omar Cornut's immediate-mode GUI library. ImGui handles all the hard problems --- text layout, widget state, input handling, GPU rendering --- and does so in a single-pass retained-mode-free architecture that maps naturally to Lux's "send JSON, render this frame" model. The 60fps render loop, the composable widget tree, and the ability to drive a full UI from a socket with no threading are all consequences of ImGui's design.

Python bindings come from imgui-bundle by Pascal Thomet, which packages ImGui, ImPlot, and several other ImGui extensions into a single pip-installable wheel with complete type stubs. imgui-bundle is what makes "install one Python package, get a GPU-accelerated UI" possible.

FastMCP provides the MCP server layer.

License

MIT