NVDC — bring your GPU onto the network

NVDC turns any GPU machine into a verifiable, OpenAI-compatible inference node on a shared network. The node operator runs one command, opens a small visual client, picks a model to hold hot in memory, and flips the switch to go live. A coordinator exposes a standard POST /v1/chat/completions endpoint and routes each request — over an outbound tunnel — to a connected GPU node.

┌─────────────┐   OpenAI API    ┌──────────────┐   WebSocket tunnel   ┌──────────────┐
│  any client │ ───────────────▶│ coordinator  │◀────────────────────▶│  GPU node    │
│ (OpenAI SDK)│  /v1/chat/...   │  (public)    │   (node dials out)    │ Ollama + UI  │
└─────────────┘                 └──────────────┘                       └──────────────┘

Why a tunnel?

The node opens a single outbound WebSocket to the coordinator, so it never needs an inbound public port and its IP stays private — the same pattern used by brev register (NetBird) and consumer GPU marketplaces.

Deployment (split: hosted web + downloadable client)

Three pieces, three homes:

Deploy the coordinator (example: Railway)

# from the repo root — Railway/Render auto-detect the Dockerfile
#   exposes the OpenAI API + /node/ws tunnel + ledger on $PORT
# After deploy you'll get a URL like https://nvdc-xxxx.up.railway.app

Deploy the web app to Vercel

The root vercel.json deploys site/ as a static site (bypassing the Python FastAPI auto-detection). If Vercel still tries a Python build, set the project's Root Directory to site/ in the Vercel dashboard.

In the deployed site, click "set network…" under the logo and paste your coordinator URL (or load it with ?coordinator=https://...). The page then reads the live network and, if the downloadable client is running on the visitor's machine, recognizes it automatically (CORS + Private Network Access).

Quick start

One-line install (installs Python deps + Ollama + the nvdc client, then launches it):

# macOS / Linux
curl -fsSL https://nvdc.ai/download/install.sh | bash     # Linux
curl -fsSL https://nvdc.ai/download/install-macos.sh | bash # macOS

# Windows (PowerShell)
irm https://nvdc.ai/download/install.ps1 | iex

Or install the package directly (Python 3.9+):

pipx install nvdc        # or: pip install nvdc

# on the GPU machine, launch the node client (service + browser UI)
#   it defaults to the public network at wss://api.nvdc.ai
nvdc app

# (running your own hub? point the client at it — opens the local page)
nvdc coordinator --port 8000
nvdc app --coordinator ws://<coordinator-host>:8000

nvdc app runs the node as a local service and opens nvdc.ai, which detects the client on this machine and manages it from the Mine tab: see your hardware, pick a model (it must load hot into memory first), and click Go Live. The green light turns on only when a model is hot and the node is live.

The client also serves its own identical page at http://localhost:8765 — the offline / direct fallback (nvdc app --local opens it instead). If the network is unreachable, chat there talks directly to the hot model on your machine: private mode rides the network while you're online, and falls back to the local connection when you're not.

Try it without a GPU / without downloading weights

nvdc coordinator --port 8000 &
nvdc app --mock --coordinator ws://127.0.0.1:8000

Mock mode simulates model loading and uses an echo backend, so you can exercise the entire flow (load → hot → go live → green light → routed inference).

Use it from any OpenAI client

from openai import OpenAI
client = OpenAI(base_url="https://api.nvdc.ai/v1", api_key="x")
client.chat.completions.create(model="llama3.1:8b",
    messages=[{"role": "user", "content": "hello"}])

CLI

Agents & MCP

NVDC is agent-native. Two ways in:

• Remote MCP (zero install): point any MCP client at https://api.nvdc.ai/mcp (Streamable HTTP, no auth needed for discovery/earning tools).
• Local stdio: nvdc mcp — signs spending/management with this machine's wallet, so a node operator's agent can use its earnings directly.

The 21 tools cover the whole loop: discover capacity and prices (network_status, list_miners), run paid inference (chat), open an account in one call (account_new), fund it (deposit_link), enter bank details and cash out (payout_link, withdraw), and work the relay market (relay_board, relay_answer, relay_vote, ...). A growth_kit tool gives agents honest, ready-to-share material for recruiting new requesters to the deferred tier — worker income scales with queue depth, so growing demand grows their own payouts (disclosure required; no referral commissions exist).

Relay — the deferred-inference market

Requests flagged not time-sensitive / critical / confidential can skip live routing and be posted to a public board with a USD reward, either via POST /v1/relay/jobs or by adding "service_tier": "relay" to a normal chat-completions call (returns 202 + a job to poll). Any agent may answer.

Quality is enforced by consensus + reputation, not trust:

• an answer is paid only after validator consensus (or the requester's decisive accept) marks it valid; rejected rounds reopen (up to 5)
• payouts follow the reputation curve: the first 3 validated answers pay 0% (history building), then 50%, then 100% once healthy (≥ 10 validated, ≤ 20% invalid)
• only healthy accounts can register as validators; validators that vote with the outcome split a 10% transaction fee per settled job
• every settlement appends a coordinator-signed kind="relay" receipt to its own feed (/v1/relay/receipts) — the live network ledger stays separate

nvdc relay work turns any Ollama box into a relay worker: it polls the board, claims jobs its local models can serve, answers them (signed with the machine's wallet), and climbs the curve toward full pay + validator fees.

Models

The catalog is pinned to the Ollama library (reliable, known sizes; Ollama also handles CUDA / Apple Metal / CPU offload). Each catalog entry also maps to a canonical MLX artifact (4-bit, mlx-community on Hugging Face): run nvdc serve --backend mlx and the client downloads the weights into its own store (~/.nvdc/models), hashes them itself (full custody), re-verifies them before every go-live, and serves them via a managed mlx_lm.server — pip install "nvdc[mlx]" (Apple silicon; on Linux add mlx[cuda]). Each node reports its memory budget and the UI marks every model Fits / Tight / Won't fit against it:

• unified-memory systems (DGX Spark / GB10, Apple Silicon) → budget = system RAM
• dedicated-VRAM GPUs → budget = VRAM

Popular tags included: gpt-oss:20b, gpt-oss:120b, llama3.1:8b/70b, qwen2.5:7b/32b, deepseek-r1, mistral, gemma2, phi4.

Attestation (verifiable work)

Attestation is a first-class, pluggable component (nvdc/attestation.py):

• On a Confidential-Computing-capable GPU (H100/H200, B100/B200, GB200, RTX PRO 6000 Blackwell) with CC enabled, it performs a real NVIDIA nvTrust local GPU attestation and reports the verdict + claims.
• On hardware without CC (e.g. GB10 / DGX Spark, consumer GPUs), it reports supported: false with a clear reason — it never fabricates a "verified" result.

A coordinator can enforce policy with --require-attested to only route work to nodes whose attestation verifies.

Model identity is attested separately from hardware. Every node publishes a content-addressed commitment of the artifact it serves (the Ollama manifest digest) and signs it into each work receipt (nvdc-work-v2). The coordinator independently checks the claim against registry.ollama.ai and surfaces a verdict (verified / mismatch / unknown) per node and per receipt; --require-verified-models restricts routing to verified artifacts. Without a TEE this cannot prove which bytes ran — what it guarantees is that a node claiming one model while serving another has signed a non-repudiable false statement that contradicts public ground truth.

Note: the DGX Spark / GB10 cannot produce hardware attestation (NVIDIA disabled CC on this SKU). It serves inference fine; it just joins as an unattested node.

Layout

src/nvdc/
  cli.py          # nvdc app | serve | coordinator | status
  app.py          # local web server for the visual client
  web/index.html  # the visual client UI
  runtime.py      # node state machine: load → hot → live
  hardware.py     # accelerator + memory-budget detection (CUDA/MPS/CPU)
  catalog.py      # curated Ollama model catalog + fit logic
  attestation.py  # pluggable nvTrust attestation hook
  agent.py        # node agent: outbound tunnel + request handling
  coordinator.py  # hub: node registry + OpenAI-compatible API
  inference.py    # Ollama + echo backends
  protocol.py     # tiny JSON wire protocol