NodeTool Core

Python Node System & Worker

NodeTool Core is the Python library that provides the node system and worker subprocess for NodeTool. The TypeScript server handles HTTP API, workflow orchestration, agents, and chat. Python handles node execution and local-compute providers (HuggingFace, MLX).

What's Here

• Node system — BaseNode, ProcessingContext, type metadata
• Worker subprocess — python -m nodetool.worker communicates with the TS server via WebSocket+MessagePack
• Provider infrastructure — Base classes and registry for local-compute providers
• Media processing — Image, audio, video conversion utilities
• Storage — Abstract storage backends (memory, file, S3)

Quick Start

Install into a conda env named nodetool. The NodeTool server looks for a Python interpreter in this order: NODETOOL_PYTHON, the active conda env when it is named nodetool, then the usual envs/nodetool locations and the desktop app's managed env. It does not find a project .venv.

conda create -n nodetool python=3.11 pandoc ffmpeg -c conda-forge
conda activate nodetool
uv pip install -e ".[dev]"   # or: pip install -e ".[dev]"

# Run tests
pytest -q

# Start a worker by hand (the TS server normally spawns it)
python -m nodetool.worker

Install node packs into the same env, for example uv pip install nodetool-huggingface. To use a different interpreter, start the server with NODETOOL_PYTHON=/path/to/python.

Writing Nodes

from nodetool.workflows.base_node import BaseNode
from nodetool.workflows.processing_context import ProcessingContext

class MyNode(BaseNode):
    """
    Brief description
    tags, keywords, for, search
    """
    input_text: str = ""

    async def process(self, context: ProcessingContext) -> str:
        return self.input_text.upper()

Nodes use ProcessingContext for media conversion (image_to_pil, audio_from_numpy, etc.), secrets (get_secret), asset storage, and progress reporting.

Architecture

TS Server (Fastify)
    ├── HTTP API, WebSocket, Auth
    ├── Workflow orchestration (DAG scheduling)
    ├── Cloud providers (OpenAI, Anthropic, Gemini, ...)
    └── Spawns Python worker subprocess
            ↕ WebSocket + MessagePack
Python Worker (this repo)
    ├── Node discovery & execution
    ├── Local providers (HuggingFace, MLX)
    └── Media processing (ffmpeg, PIL, numpy)

Worker Protocol

The worker speaks a msgpack message protocol to the TS server (bridge protocol v6, nodetool.worker.BRIDGE_PROTOCOL_VERSION). Two transports carry the same messages:

• WebSocket (default): python -m nodetool.worker --host 0.0.0.0 --port 8787. Without --port the worker binds a free port (NODETOOL_WORKER_PORT, default 0). Do not use 7777, the NodeTool server's own default port. Each message is one binary msgpack frame. On startup the worker prints NODETOOL_WORKER_PORT=<port> to stdout (the only thing on stdout). If NODETOOL_WORKER_TOKEN is set, the opening handshake must carry Authorization: Bearer <token> (constant-time compare, rejected with 401 before any frame); unset means open, for local/dev use.
• stdio (--stdio): same msgpack payloads with 4-byte big-endian length-prefixed framing over stdin/stdout, for parent processes that spawn the worker directly.

Frames are capped at 256 MiB by default (NODETOOL_BRIDGE_MAX_FRAME_SIZE). Unknown msgpack extension types decode to None rather than erroring. Binary data (images, audio, model files) travels as native msgpack bin values — there is no base64.

Message envelope

Every message is a map: {"type": string, "request_id": string, "data": object}. Requests carry a unique request_id; every frame the worker sends back echoes it, so concurrent requests multiplex safely over one connection. The worker replies with frames of type:

cancel ({type: "cancel", request_id: <id-of-in-flight-request>}) requests cooperative cancellation of a running request; it produces no reply of its own — the cancelled request emits its own terminal frame.

Core messages

For execute, fields are the node's property values; blobs maps field names to binary inputs (written to temp files and resolved as asset refs); secrets are forwarded to the node's context. Binary outputs come back in the result's blobs map.

Provider messages (provider.*)

Local-compute providers (HuggingFace local, MLX) registered in this Python environment:

provider.stream and provider.tts honor cancel.

Model cache messages (models.*)

HuggingFace cache management on the worker's HF_HOME:

Tokens are resolved on the worker; client-supplied tokens are ignored. Downloads honor cancel.

ComfyUI proxy messages (comfy.*)

When a ComfyUI server is co-located (COMFYUI_URL, advertised via worker.status.comfy), the worker proxies it — see docs/comfy-proxy.md for frame-level shapes:

Input blobs are uploaded to ComfyUI and spliced into the workflow wherever a "blob:<key>" placeholder appears; file outputs are fetched from ComfyUI and returned as result blobs. comfy.execute and comfy.models.download honor cancel.

Protocol version history

The TS bridge declares the minimum version it can speak; a worker reporting a lower protocol_version in discover/worker.status is rejected.

External Node Packages

• nodetool-huggingface — HuggingFace model integrations + local provider
• nodetool-mlx — Apple Silicon optimized nodes + MLX provider (macOS on Apple Silicon only)
• nodetool-wan2gp — Wan2GP video generation through a Wan2GP server

Replicate, FAL and ElevenLabs integrations are TypeScript packs in the NodeTool repository.

Development

Use the nodetool env from the quick start:

pytest -q          # tests
ruff check .       # lint
make typecheck     # ty type check (uses uv run)

uv sync, uv run and the make targets use a separate .venv pinned by uv.lock, as CI does (uv sync --locked --all-extras --dev). After changing dependencies in pyproject.toml, run uv lock and commit uv.lock.

License

AGPL License

Learn More

• NodeTool Website
• Discord Community