gpumesh 0.3.0

Borrow your friends' GPUs: a terminal-based distributed compute mesh in pure Python

gpumesh

Borrow your friends' GPUs. A terminal-based distributed compute mesh written in pure Python (stdlib only — torch, psutil, pyngrok are optional extras).

You have ML work to run but a weak laptop. Your friend has a strong GPU sitting idle. gpumesh turns any group of machines into a small compute cluster: one machine coordinates, any number of machines join with a single command, and work is split between them proportionally to how fast each machine is.

 your laptop (coordinator)              friend's laptop (worker)
┌───────────────────────┐    HTTP/JSON  ┌────────────────────────┐
│  job queue            │◄─────────────►│  hardware probe        │
│  capability scheduler │   (optional   │  matmul benchmark      │
│  SQLite (WAL)         │    ngrok      │  sandboxed subprocess  │
│  heartbeats + reaper  │    tunnel)    │  executor              │
└──────────▲────────────┘               └────────────────────────┘
           │ submit / status
       client CLI

Quick start

pip install -e .

Machine 1 — start the coordinator:

gpumesh serve --port 8000
# prints a token and a ready-made join command, e.g.
#   gpumesh join http://192.168.1.5:8000 --token Kv3xP9qL2mNa

Add --public (with pip install pyngrok) to get a public URL friends can reach over the internet.

Machine 2..N — join as a worker (one command, as promised):

gpumesh join http://192.168.1.5:8000 --token Kv3xP9qL2mNa

The worker probes its hardware (CUDA GPU → Apple Silicon → CPU), benchmarks itself with a matmul, and starts pulling tasks. Your own laptop can join too — then both machines compute in parallel.

Submit work:

gpumesh submit examples/grid_search.py --payloads examples/payloads.json \
    --url http://192.168.1.5:8000 --token Kv3xP9qL2mNa --wait

A job is a Python script plus a JSON list of payloads (shards). Each payload becomes one task; an optional "cost" field marks how heavy it is. Heavy tasks go to fast machines, light tasks to slow ones.

Watch the mesh:

gpumesh workers --url ... --token ...
gpumesh status JOB_ID --url ... --token ...

Writing your own task script

The contract is two lines of glue:

import json, sys
payload = json.load(sys.stdin)        # your shard's parameters
# ... do the work (os.environ["GPUMESH_DEVICE"] tells you cuda/mps/cpu) ...
print(json.dumps({"answer": 42}))     # result = last stdout line, as JSON

Anything data-parallel fits: hyperparameter search, batch inference, cross-validation folds, rendering chunks, brute-force search shards.

How the scheduler splits work

1. Every worker benchmarks itself at join time → a GFLOP/s score.
2. Pending tasks are sorted by cost.
3. When a worker asks for work, its score percentile among live workers picks the matching percentile of task costs — the strongest machine gets the heaviest remaining task, the weakest gets the lightest.
4. Tasks are leased, not given away: if a worker dies (heartbeats stop) or the lease expires, the reaper re-queues the task for someone else (max 3 attempts, then marked failed).

Design principles

• Networking — Hand-rolled JSON-over-HTTP protocol on http.server with no external frameworks. Worker heartbeats, poll-based task leasing that survives flaky connections, shared-token auth, and optional ngrok tunneling for NAT traversal.
• Database — SQLite in WAL mode with foreign keys, indexes, and atomic lease acquisition via conditional UPDATE ... WHERE status='pending' to prevent duplicate task assignments. Transactions protect multi-row writes.
• Process isolation — Every task runs in a fresh subprocess in its own process group. Timeouts kill the entire process tree; POSIX RLIMIT_CPU caps runaway loops. The coordinator uses threads with a lock around the shared DB connection.
• Fault tolerance — Capability-based scheduling, lease/heartbeat failure detection, idempotent re-queue with bounded retries, and a pull-based model where workers fetch work instead of the coordinator pushing it.
• Task parallelism — Independent work units are sharded across machines, not model tensors. This avoids internet latency bottlenecks and matches how production serverless GPU platforms operate.

Security

Workers execute code submitted to the coordinator — that's the core functionality. Only share your coordinator URL and token with people you trust. Treat the token like a password and don't leave a public tunnel running unattended. The subprocess sandbox limits accidents, not determined attackers.

Security features in v0.2.0:

• Token hashing with SHA-256 and random salt
• Rate limiting (5 attempts per 5 minutes, 15 minute lockout)
• Optional IP allowlist for additional access control

Layout

gpumesh/
  cli.py         command line entry point (serve / join / submit / status / workers)
  server.py      coordinator: threaded HTTP JSON API + lease reaper
  db.py          SQLite layer: workers, jobs, tasks, atomic leasing
  worker.py      agent loop: register, heartbeat, lease, execute, report
  sandbox.py     subprocess isolation: timeouts, process-group kill, rlimits
  capability.py  hardware probe + matmul benchmark -> capability score
  client.py      job submission and status polling
  tunnel.py      optional ngrok public URL
examples/
  grid_search.py hyperparameter-search demo task (pure Python)
  payloads.json  six shards with varying cost