rustygate

A Jupyter kernel gateway as one binary: kernels, terminals, files, search, and per-cell notebook operations, over the same REST and websocket API shapes jupyter_server uses.

Running

rustygate [--host H] [--port N] [--token T | --token-file PATH] [--trust-loopback] [--root DIR] [--tls] [--autorun DIR]

The defaults are 127.0.0.1, port 8787, no auth, and the working directory as the files root. With a token set, requests carry Authorization: Bearer T; Jupyter's Authorization: token T form and ?token=T are also accepted. --token-file reads the token without putting it in the process arguments. --trust-loopback lets clients whose actual TCP peer is loopback omit the token. Do not use it behind an untrusted same-host reverse proxy, since the proxy is the TCP peer. Forwarded headers never affect authentication. --tls serves https and wss from a self-signed certificate generated at startup; clients opt out of verification (jupyasyncclient's verify=False).

The Python helper rustygate.tools.start_gateway() starts a background gateway with inherited stdout and stderr. Gateway and kernel startup errors appear in the calling process's console.

Working directories

Notebook-bound kernels default to the notebook's parent directory under the served root. Folder-bound terminals default to their folder. Explicit cwd overrides either binding; relative values use the served root. Restart recomputes a kernel's default cwd from its current binding. See DEV.md for unbound processes and path rules.

GET /api/kernels/{kid}/realpath?path=... returns an on-disk path as a JSON string. Empty and relative paths use the bound notebook's parent, regardless of the kernel's cwd. A leading slash selects the gateway root. The target need not exist. Notebook-relative paths require a notebook binding; root-qualified paths need only an existing kernel. Existing symlinks and .. resolve within the root; escapes and dangling symlinks are rejected. Notebook bindings separately require one canonical spelling, without symlink aliases.

Notebook state and connections

Clients sharing a notebook share its kernel and authoritative notebook state. Accepted edits broadcast to every subscriber, including their author. Saving is debounced; contents reads, copies, search and command execution first save pending changes in their file or directory scope, then operate on disk normally.

POST /api/kernels with a path returns its existing non-dead kernel unchanged (200), ignoring startup options, or starts a new kernel (201). A concurrent creation still starting returns 409. Without a path, each request starts a new kernel.

A kernel id identifies the logical kernel across rename, restart and failed-restart retry. GET/POST /api/kernels/{kid}/cells operates on its current notebook binding. PATCH /api/kernels/{kid} with {"path":"new.ipynb"} renames that notebook. Deletion and new creation produce a new id. Path-addressed file and cell operations remain available for unopened notebooks.

Cell reads support section=ID for a heading and its descendants, ancestors=ID for enclosing headings, and before=ID or after=ID for every cell on one side of a cell. All use ordinary cell IDs and ignore collapse flags. fields=id returns IDs in cell dictionaries; fields=id,source returns IDs and sources. Omit fields for full cells without attachments, or use fields=* for all cell fields including attachments. fields=*,meta also returns notebook metadata. See DEV.md for heading detection and selector composition.

max_tokens=N keeps the newest cells whose approximate token counts total at most N. up=NAME adds inherited, the cells of each notebook called NAME in the notebook's folder and its ancestor folders, root first. refs=true adds refs, the $`expr`, !`cmd` and &`name` references in the returned cells. See DEV.md for these contracts.

Notebook settings use the same ordered ops: {"ops":[{"op":"meta","metadata":{"solveit":{"mode":"learn"}}}]} merge-patches notebook metadata without transferring or replacing cells and outputs. Accepted metadata broadcasts carry the resulting metadata, so subscribers need no follow-up read.

File and cell edits use last-accepted-edit-wins semantics, without hashes or stale-write checks. File writes and copies can require a new destination with overwrite=false; renames never overwrite. The watcher reloads valid external changes when the file's mtime changes.

Kernel traffic uses bounded queues with backpressure before notebook updates. A slow frontend's full queue closes its whole connection rather than silently dropping individual messages. Disconnected messages are not replayed: reconnecting clients load current notebook state. Kernel execution and persistence continue without connected clients.

Startup notebooks

--autorun AUTORUN runs the direct *.ipynb children of AUTORUN under the files root once per gateway startup. Each notebook gets its own path-bound kernel, with its parent directory as the working directory. Nonempty code cells run in notebook order; outputs persist normally. Other cell types and CRAFT inheritance are not processed. Shared initialization belongs in explicit imports in the notebook.

Notebooks run independently in the background, without delaying HTTP readiness. A cell error stops the remaining cells in that notebook; errors are logged and its kernel remains available for inspection. Other notebooks and the gateway keep running. A missing directory is empty; unreadable directories or malformed notebooks produce an error. Attaching a client or restarting an individual kernel does not rerun AUTORUN. Omit the flag to disable it. rustygate service install accepts the same flag.

Running as a service

rustygate service installs and controls a user service using launchd on macOS and systemd on Linux:

rustygate service install --root ~/git --workdir ~/git/project
rustygate service status
rustygate service restart
rustygate service stop
rustygate service start
rustygate service uninstall

install starts the service immediately; running it again replaces the configuration. Add --host 0.0.0.0 for direct network access, --token-file ~/.config/rustygate/token for authentication, --trust-loopback when same-host clients should not need that token, and --tls when wanted. A missing token file is generated with user-only permissions. --env-file sources environment variables inherited by kernels. The service PATH is the rustygate binary's own directory (the venv bin, under a pip install) plus the standard system directories, so kernels find python despite the minimal launchd and systemd defaults. The default loopback service can instead be reached remotely through an SSH tunnel.

Kernels

POST /api/kernels starts the kernel that name selects. A name is the bundled luau, or an installed Jupyter kernelspec. pip install ipymini installs the py kernelspec (Python/IPython). py is the default when a request has neither name nor argv. pip install basedpl installs apl. A name cannot be combined with a custom argv. An unknown name returns 400 with the available names.

GET /api/kernelspecs lists the available names with their kernel.json contents, in jupyter_server's format. GET /api/kernelspecs/{name} returns one. The gateway reads the kernelspec directories on each request. A kernel installed while the gateway runs needs no gateway restart.

The gateway searches the directories that jupyter kernelspec list searches, in the same order, then ~/.ipython/kernels. The first kernelspec of a name wins. The environment directories are share/jupyter beside the bin/ that holds the rustygate binary, then under $VIRTUAL_ENV, then under $CONDA_PREFIX. A binary outside any Python environment finds an activated environment's kernels that way. An interrupt follows the kernelspec's interrupt_mode. The gateway sends SIGINT, or a control-channel interrupt_request when the mode is message.

See DEV.md for the search order and the kernel.json fields the gateway uses.

Built-in Luau kernel

Start with help() in a Luau cell (or create(kernel="luau") then exec(code="help()") through MCP). It returns a bundled quick-start guide and API index; help("ex.edit_file") gives function details, using the same documentation as Jupyter inspection. Inside a longer cell use print(help()) to display it. No external docs/install needed.

Create with POST /api/kernels and {"name":"luau"} (optionally cwd and path). The gateway starts its own executable as rustygate native-kernel -f CONNECTION_FILE. No Python, Lua installation, or external search command is needed. luau is built in and wins over a kernelspec of that name.

Native APIs include rg.search, rg.find, fs.read_text, ex.edit_text, file/cell viewing and editing, os.execute, io.popen, print, and display. For example, in successive cells:

hits = rg.search("TODO", {root=".", exts={"rs"}, max_results=100})

#hits.matches

original = fs.read_text("notes.txt")
edited = ex.edit_text(original, "%s/TODO/DONE/g")
return edited.text

ex.edit_text edits private in-memory text. To edit files or notebook cells:

print(ex.view_file("notes.txt")) -- use addresses from this view
-- replace ADDRESS with a verified address from the view
-- ex.edit_file("notes.txt", {{ADDRESS, "s", "TODO", "DONE"}})
-- ex.edit_cell("analysis.ipynb", "cell-id-prefix", {{ADDRESS, "c", "new source"}})

File commands use arrays of strings (nested arrays for global subcommands), like Python command tuples. path:address and path.ipynb:cellid:address prefixes allow cross-file/cell moves and copies. ex.edit_cell also accepts a compact script. Options are {inplace=false} for preview and {sw=4} for shift width; writes are enabled by default. ex.edit_file returns {files, changed, diff}; ex.edit_cell returns one result. Every command is validated/applied before any writes; this is not an atomic multi-file transaction. Notebook metadata, outputs, source form and trailing newlines are preserved.

local success, kind, code = os.execute("printf hello | tr a-z A-Z")
local p = assert(io.popen("git status --short"))
local output = p:read("*a")
local success, kind, code = p:close()
return output

Both launch /bin/sh -c in the kernel cwd/environment; Bash-specific syntax requires explicitly calling bash -c. os.execute publishes stdout/stderr and returns Lua 5.4-style success/nil, exit/signal, and code. No-command os.execute() checks shell availability. Commands are noninteractive: stdin is closed unless using io.popen(command, "w"). Programs you invoke must exist, but Python or a separate Lua install is not needed.

Pipes support read, lines, write, flush, close, and seek (which returns a pipe-not-seekable error). Reads accept byte counts or l, L, a, n (also *l, etc.), with multiple formats and binary-safe strings. Writes are unbuffered and return the handle; close returns the command status. This is not the full Lua io library: no io.open, default streams, or setvbuf. io.type and io.close(pipe) are available. stderr and write-mode stdout are forwarded during pipe operations. A pipe can persist across cells; close it explicitly. GC, interruption and kernel shutdown terminate outstanding processes; an interrupt closes all kernel-owned pipes.

Search records carry path, line number, lnhash and match spans. Search/discovery return all results by default; an optional max_results truncates them and sets complete=false when results are omitted. There are no imposed byte or result-count caps. This is trusted local scripting, not an OS sandbox or memory-bounded execution.

fs.read_text uses a standard UTF-8 file read with normal filesystem/encoding errors and no custom size limit. Interrupts are checked before and after the read; they do not abort an OS call in progress.

Globals/functions persist; local variables remain cell-local. A cell may be a whole expression or an ordinary chunk with an explicit return. Stream output, MIME display, errors, user expressions, basic host-API completion/help, and the tail retrieval of session input history are supported. display takes a MIME bundle, for example display({['text/markdown']='**hello**'}).

SIGINT and control-channel interruption share an execution-specific token with the VM and native operations. Text edits run normally, with interrupt checks before and after the call; an interrupt waits for the edit to finish and discards its result (file writes that completed are not rolled back). Shell process groups are killed and their direct children reaped on interruption. Search workers are cancelled and joined before execution completes. Earlier statements are not rolled back. Individual regex/compiler/filesystem calls and allocation/free remain cooperative boundaries, not hard real-time preemption. Restart/delete still terminate the child independently.

This initial implementation has no IPython magics, debugger, typechecker/LSP, subshells, or alternate interpreter backend. MCP uses create(kernel="luau") followed by language-independent exec calls. Standalone executable release artifacts and Linux validation are follow-up work. The design and experiment record are in meta/luau.md.

MCP

Choose a kernel, then execute code in it:

create(kernel="apl")
exec(code="avg←+/÷≢ ⋄ avg 2 4 9")

kernel takes the names POST /api/kernels takes: the bundled luau, and the installed kernelspecs such as py (Python/IPython, installed by ipymini) and apl (installed by basedpl). The create tool's schema lists the names found when the client lists the tools. New kernels require an explicit name. exec never creates or switches kernels. create(dlgname="work", kernel="luau") binds a kernel to a dialog. Omit kernel only to reuse an existing binding. An explicit name must match that binding. Omit dlgname to create an unbound kernel.

Creation and selection return kernel-specific guidance, including Luau's help() entry point. Creation, selection, and listing report both the implementation and the language from kernel_info_reply. Python startup/inspectors run only for Python kernels, including on restart. Clikernel forwards the tools and arguments directly, including creation on named gateways.

POST /mcp is a stateful Streamable HTTP MCP server for Claude Code and other MCP clients. Its tools are list_kernels, create, use_kernel, delete_kernel, exec, restart, and interrupt. Code runs exactly as written; Python/IPython magics such as %%bash work in Python kernels. The MCP session remembers its current kernel, so no id is needed between calls. Call create or use_kernel before executing. Ending the session (DELETE /mcp) stops every kernel it created with autoclose, enabled by default; autoclose=false keeps a new kernel. Reusing or selecting an existing kernel never claims ownership. An execution may name another existing binding with dlgname for that call without changing selection. Clients may send creation defaults (cwd, env, Python startup code, quiet) in initialize params; see DEV.md.

POST /mcp/{path} is the stateless alternative for one dialog. The URL-decoded relative path fixes the binding. It exposes create(kernel?), exec(code), and interrupt(). Creation requires an implementation when unbound; otherwise it reuses the kernel, checking any explicitly requested implementation. Execution and interrupt require an existing binding. With no owning session, these kernels persist until deleted or the gateway stops. Executions use priority 1, ahead of queued normal-priority notebook runs without preempting an active cell.

Kernels actually spawned through MCP receive RUSTYGATE_LAUNCHER=mcp, available during their startup profiles and preserved across restart. Selecting or get-or-creating a kernel that was already running does not alter its environment. See DEV.md for the tool and execution contract.