nvdc 0.2.5

Bring your GPU onto the network: one command turns a GPU into a verifiable, OpenAI-compatible inference node.

NVDC — bring your GPU onto the network

NVDC turns any GPU machine into a verifiable, OpenAI-compatible inference node on a shared network. The node operator runs one command, opens a small visual client, picks a model to hold hot in memory, and flips the switch to go live. A coordinator exposes a standard POST /v1/chat/completions endpoint and routes each request — over an outbound tunnel — to a connected GPU node.

┌─────────────┐   OpenAI API    ┌──────────────┐   WebSocket tunnel   ┌──────────────┐
│  any client │ ───────────────▶│ coordinator  │◀────────────────────▶│  GPU node    │
│ (OpenAI SDK)│  /v1/chat/...   │  (public)    │   (node dials out)    │ Ollama + UI  │
└─────────────┘                 └──────────────┘                       └──────────────┘

Why a tunnel?

The node opens a single outbound WebSocket to the coordinator, so it never needs an inbound public port and its IP stays private — the same pattern used by brev register (NetBird) and consumer GPU marketplaces.

Deployment (split: hosted web + downloadable client)

Three pieces, three homes:

Component	Where it runs	Notes
Coordinator (nvdc coordinator)	A persistent host (Railway / Render / Fly.io / VM)	Needs long-lived WebSockets + in-memory state. Not Vercel serverless. A Dockerfile + Procfile are included.
Web app (site/)	Vercel (static)	Mirrors the client UI: Home/Chat/Network read from the coordinator (CORS); Mine shows a download CTA + live market figures, and lights up with real data if the client is running locally.
Downloadable client (nvdc app)	The miner's GPU box	The full app from above — detects the GPU, mines, holds the signing identity.

Deploy the coordinator (example: Railway)

# from the repo root — Railway/Render auto-detect the Dockerfile
#   exposes the OpenAI API + /node/ws tunnel + ledger on $PORT
# After deploy you'll get a URL like https://nvdc-xxxx.up.railway.app

Deploy the web app to Vercel

The root vercel.json deploys site/ as a static site (bypassing the Python FastAPI auto-detection). If Vercel still tries a Python build, set the project's Root Directory to site/ in the Vercel dashboard.

In the deployed site, click "set network…" under the logo and paste your coordinator URL (or load it with ?coordinator=https://...). The page then reads the live network and, if the downloadable client is running on the visitor's machine, recognizes it automatically (CORS + Private Network Access).

Quick start

One-line install (installs Python deps + Ollama + the nvdc client, then launches it):

# macOS / Linux
curl -fsSL https://nvdc.ai/download/install.sh | bash     # Linux
curl -fsSL https://nvdc.ai/download/install-macos.sh | bash # macOS

# Windows (PowerShell)
irm https://nvdc.ai/download/install.ps1 | iex

Or install the package directly (Python 3.9+):

pipx install nvdc        # or: pip install nvdc

# on the GPU machine, launch the visual client
#   it defaults to the public network at wss://api.nvdc.ai
nvdc app

# (running your own hub? point the client at it)
nvdc coordinator --port 8000
nvdc app --coordinator ws://<coordinator-host>:8000

Then in the browser UI: see your hardware, pick a model (it must load hot into memory first), and click Go Live. The green light turns on only when a model is hot and the node is live.

Try it without a GPU / without downloading weights

nvdc coordinator --port 8000 &
nvdc app --mock --coordinator ws://127.0.0.1:8000

Mock mode simulates model loading and uses an echo backend, so you can exercise the entire flow (load → hot → go live → green light → routed inference).

Use it from any OpenAI client

from openai import OpenAI
client = OpenAI(base_url="https://api.nvdc.ai/v1", api_key="x")
client.chat.completions.create(model="llama3.1:8b",
    messages=[{"role": "user", "content": "hello"}])

CLI

Command	What it does
nvdc app	Launch the visual node client (web UI)
nvdc serve	Headless node: bring this GPU onto the network
nvdc coordinator	Run the public hub + OpenAI-compatible API
nvdc status	Print local GPU + attestation status as JSON

Models

The catalog is pinned to the Ollama library (reliable, known sizes; Ollama also handles CUDA / Apple Metal / CPU offload). Each catalog entry also maps to a canonical MLX artifact (4-bit, mlx-community on Hugging Face): run nvdc serve --backend mlx and the client downloads the weights into its own store (~/.nvdc/models), hashes them itself (full custody), re-verifies them before every go-live, and serves them via a managed mlx_lm.server — pip install "nvdc[mlx]" (Apple silicon; on Linux add mlx[cuda]). Each node reports its memory budget and the UI marks every model Fits / Tight / Won't fit against it:

• unified-memory systems (DGX Spark / GB10, Apple Silicon) → budget = system RAM
• dedicated-VRAM GPUs → budget = VRAM

Popular tags included: gpt-oss:20b, gpt-oss:120b, llama3.1:8b/70b, qwen2.5:7b/32b, deepseek-r1, mistral, gemma2, phi4.

Attestation (verifiable work)

Attestation is a first-class, pluggable component (nvdc/attestation.py):

• On a Confidential-Computing-capable GPU (H100/H200, B100/B200, GB200, RTX PRO 6000 Blackwell) with CC enabled, it performs a real NVIDIA nvTrust local GPU attestation and reports the verdict + claims.
• On hardware without CC (e.g. GB10 / DGX Spark, consumer GPUs), it reports supported: false with a clear reason — it never fabricates a "verified" result.

A coordinator can enforce policy with --require-attested to only route work to nodes whose attestation verifies.

Model identity is attested separately from hardware. Every node publishes a content-addressed commitment of the artifact it serves (the Ollama manifest digest) and signs it into each work receipt (nvdc-work-v2). The coordinator independently checks the claim against registry.ollama.ai and surfaces a verdict (verified / mismatch / unknown) per node and per receipt; --require-verified-models restricts routing to verified artifacts. Without a TEE this cannot prove which bytes ran — what it guarantees is that a node claiming one model while serving another has signed a non-repudiable false statement that contradicts public ground truth.

Note: the DGX Spark / GB10 cannot produce hardware attestation (NVIDIA disabled CC on this SKU). It serves inference fine; it just joins as an unattested node.

Layout

src/nvdc/
  cli.py          # nvdc app | serve | coordinator | status
  app.py          # local web server for the visual client
  web/index.html  # the visual client UI
  runtime.py      # node state machine: load → hot → live
  hardware.py     # accelerator + memory-budget detection (CUDA/MPS/CPU)
  catalog.py      # curated Ollama model catalog + fit logic
  attestation.py  # pluggable nvTrust attestation hook
  agent.py        # node agent: outbound tunnel + request handling
  coordinator.py  # hub: node registry + OpenAI-compatible API
  inference.py    # Ollama + echo backends
  protocol.py     # tiny JSON wire protocol