gpumesh

Borrow your friends' GPUs. A distributed compute mesh that lets you share GPU power across machines on your network — with a single decorator, one CLI command, or a Python API.

What is gpumesh?

gpumesh turns multiple machines into a single, unified GPU compute pool. You start a coordinator on one machine, join workers from other machines (laptops, desktops, servers — anything with Python), and then run code across all of them as if they were one device.

 Machine A (coordinator)       Machine B (worker)       Machine C (worker)
 ┌─────────────────────┐      ┌──────────────────┐     ┌──────────────────┐
 │  RTX 4090           │      │  RTX 3080        │     │  T4              │
 │  Score: 120.5       │◄────►│  Score: 85.2     │◄───►│  Score: 12.0     │
 │                     │      │                  │     │                  │
 │  @accelerate(mesh)  │      │  receives task   │     │  receives task   │
 │  def train():       │      │  runs train()    │     │  runs train()    │
 └─────────────────────┘      └──────────────────┘     └──────────────────┘
          │
          ▼
    Results collected automatically

Use cases:

• Hyperparameter search across multiple GPUs
• Data preprocessing sharded across machines
• Model training on a pool of consumer GPUs
• Any embarrassingly parallel workload

Features

Transparent Acceleration

@accelerate(mesh)
def train(lr, epochs):
    return {"accuracy": 0.95}

# Single call → best local device
result = train(lr=0.01, epochs=100)

# Batch call → spread across ALL mesh devices
results = train.map([
    {"lr": 0.01, "epochs": 100},
    {"lr": 0.05, "epochs": 200},
])

Hardware Selection

Target specific GPU types:

@accelerate(mesh, gpu="A100")
def train(model):
    return model.cuda().forward(x)

Resource Specs

Declare what your task needs:

@accelerate(mesh, cores=8, memory="16GB", timeout=300)
def heavy_computation(data):
    return processed

Auto Device Placement

PyTorch models are automatically placed on the best device:

@accelerate(mesh)
def train_model(model, data):
    # model is automatically moved to the best GPU
    return model(data)

Fault Tolerance

• Dead workers are detected and tasks are re-queued
• Straggler workers are deprioritized
• Graceful fallback to local execution if mesh is unavailable
• Crash diagnostics on worker failures

Smart Scheduling

• Benchmark scoring — each worker gets a 0-100 performance score
• Memory-aware — tasks routed to workers with enough VRAM
• Straggler deprioritization — slow workers get fewer tasks
• TTL expiry — stale workers are automatically pruned

One-Line Setup

gpumesh setup    # Detects hardware, guides you through coordinator/worker
gpumesh quickjoin  # One-click: detect GPU and join mesh

Installation

Basic install

pip install gpumesh

With optional extras

pip install gpumesh[gpu]       # GPU detection + CUDA benchmarks (requires torch)
pip install gpumesh[tunnel]    # ngrok for public URLs
pip install gpumesh[sysinfo]   # System info (psutil)
pip install gpumesh[notebook]  # DataFrame support (pandas)
pip install gpumesh[ui]        # Beautiful setup wizard (rich + questionary)
pip install gpumesh[all]       # Everything above

Requirements

• Python 3.9+
• cloudpickle (automatically installed)
• Optional: PyTorch for GPU detection and CUDA benchmarks

Quick Start

Step 1: Start a coordinator (one machine)

gpumesh setup

The wizard will:

1. Detect your hardware (CPU cores, GPU model, VRAM)
2. Ask if you want to be a coordinator or worker
3. Generate a token and show connection info
4. Display a live radar of connected workers

Or start directly:

gpumesh serve --port 8000 --token mysecret

Step 2: Join a worker (another machine)

gpumesh setup

Choose Worker and enter the coordinator's URL and token. Or:

gpumesh join http://coordinator-ip:8000 --token mysecret

Or use quickjoin (auto-detects GPU):

gpumesh quickjoin http://coordinator-ip:8000 --token mysecret

Step 3: Use your mesh

Option A: The @accelerate decorator (recommended)

from gpumesh import GPUMesh, accelerate

mesh = GPUMesh("http://coordinator:8000", token="mysecret")

@accelerate(mesh)
def train(lr, epochs):
    # Your code here — runs on all connected GPUs automatically
    return {"accuracy": 0.95}

# Single call → best local device
result = train(lr=0.01, epochs=100)

# Batch call → spread across all mesh devices
results = train.map([
    {"lr": 0.01, "epochs": 100},
    {"lr": 0.05, "epochs": 200},
])

Option B: The Python API

from gpumesh import GPUMesh

mesh = GPUMesh("http://coordinator:8000", token="mysecret")

# List connected workers
workers = mesh.workers()
print(workers)
# [{'id': 'w1', 'device': 'cuda', 'device_name': 'RTX 3080', 'score': 85.0}]

# Distribute a function across all workers
results = mesh.distribute(
    function=train_model,
    params=[
        {"lr": 0.01, "epochs": 100},
        {"lr": 0.05, "epochs": 200},
    ],
)

# Convert to DataFrame (optional)
df = mesh.results_to_dataframe(results)

Option C: CLI job submission

# Submit a Python script with payloads
gpumesh submit train.py --payloads payloads.json --wait

# Check status
gpumesh status JOB_ID

# Cancel if needed
gpumesh cancel JOB_ID

CLI Commands

Server & Connection

Job Management

Monitoring

Useful Flags

Python API

GPUMesh Client

from gpumesh import GPUMesh

mesh = GPUMesh("http://coordinator:8000", token="mysecret")

Listing Workers

workers = mesh.workers()
# [
#     {'id': 'w1', 'device': 'cuda', 'device_name': 'RTX 3080', 'hostname': 'laptop-a', 'score': 85.0, 'alive': True},
#     {'id': 'w2', 'device': 'cuda', 'device_name': 'T4', 'hostname': 'server-b', 'score': 12.0, 'alive': True},
# ]

Distributing Functions

def train_model(lr, epochs):
    # Your training code
    return {"accuracy": 0.95, "lr": lr}

results = mesh.distribute(
    function=train_model,
    params=[
        {"lr": 0.01, "epochs": 100},
        {"lr": 0.05, "epochs": 200},
        {"lr": 0.1, "epochs": 300},
    ],
    timeout=600,  # optional, default 300s
)
# [{'accuracy': 0.95, 'lr': 0.01}, {'accuracy': 0.95, 'lr': 0.05}, ...]

Device Access

# List all devices (local + remote)
devices = mesh.devices()
# [{'index': 0, 'hostname': 'laptop-a', 'device': 'cuda', 'device_name': 'RTX 3080', 'score': 85.2, 'status': 'alive'}]

# Count GPUs
count = mesh.device_count()  # 2

# Total compute score
total = mesh.total_score()  # 205.7

# Auto-pick best device
best = mesh.auto_device()

Job Management

# Submit a raw job
job_id = mesh.submit(name="preprocess", script="process.py", payloads=[{"file": "data.csv"}])

# Check status
status = mesh.status(job_id)
# {'finished': False, 'counts': {'pending': 0, 'running': 2, 'done': 1}, 'tasks': [...]}

# Convert results to DataFrame
df = mesh.results_to_dataframe(results)

Starting Workers from Python

# Start a coordinator (non-blocking, runs in background thread)
GPUMesh.start_coordinator(port=8000, token="mysecret")

# Join as a worker (non-blocking)
info = GPUMesh.add_worker("http://coordinator:8000", token="mysecret")

@accelerate Decorator

The @accelerate decorator makes your mesh resources transparent to your code.

Pattern 1: Basic Usage

from gpumesh import GPUMesh, accelerate

mesh = GPUMesh("http://coordinator:8000", token="mysecret")

@accelerate(mesh)
def preprocess(chunk_id, data_path):
    import pandas as pd
    df = pd.read_parquet(data_path)
    return {"chunk": chunk_id, "rows": len(df)}

# Single call → runs locally on best device
result = preprocess(chunk_id=0, data_path="data.parquet")

# Batch call → spreads across ALL mesh devices
results = preprocess.map([
    {"chunk_id": 0, "data_path": "part0.parquet"},
    {"chunk_id": 1, "data_path": "part1.parquet"},
])

Pattern 2: Hardware Selection

@accelerate(mesh, gpu="A100")
def train(model):
    return model.cuda().forward(x)

Pattern 3: Resource Specs

@accelerate(mesh, cores=8, memory="16GB", timeout=300)
def heavy_computation(data):
    return processed

Pattern 4: Global Install (Import Hook)

from gpumesh import GPUMesh, accelerate

mesh = GPUMesh("http://coordinator:8000", token="mysecret")
accelerate.install(mesh)  # Set global mesh

# Now all @accelerate functions auto-use the mesh
@accelerate  # No parentheses needed
def train(lr, epochs):
    return {"accuracy": 0.95}

Pattern 5: Binding to a Device

@accelerate(mesh)
def predict(x):
    return model(x)

# Bind to a specific device
gpu_predict = predict.to("cuda")
result = gpu_predict(x)

Pattern 6: Mesh Fallback

# If the mesh is unreachable, @accelerate falls back to local execution
@accelerate(mesh)
def train(lr, epochs):
    return {"accuracy": 0.95}

# This works even if the coordinator is down — runs locally
result = train(lr=0.01, epochs=100)

Network Options

LAN (Default)

No setup required. Workers discover the coordinator automatically on the same network via UDP broadcast.

# Coordinator
gpumesh serve --port 8000

# Worker (on another machine)
gpumesh join http://192.168.1.10:8000 --token mysecret

Tailscale

Encrypted tunnel across the internet. Both machines need Tailscale installed.

# Coordinator
gpumesh serve --port 8000 --tailscale

# Worker
gpumesh join http://tailscale-ip:8000 --token mysecret

ngrok

Public URL for demos and testing.

# Coordinator
gpumesh serve --port 8000 --public
# Prints: ngrok tunnel → https://abc123.ngrok.io

# Worker
gpumesh join https://abc123.ngrok.io --token mysecret

How It Works

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Coordinator                             │
│                                                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐    │
│  │ Job Queue │  │ Task DB  │  │ Workers  │  │ Events   │    │
│  │          │  │ (SQLite) │  │ Registry │  │ Log      │    │
│  └────┬─────┘  └──────────┘  └────┬─────┘  └──────────┘    │
│       │                           │                          │
│       └───────────┬───────────────┘                          │
│                   │                                          │
│          HTTP API (port 8000)                                │
└───────────────────┼──────────────────────────────────────────┘
                    │
        ┌───────────┼───────────┐
        │           │           │
   ┌────▼────┐ ┌────▼────┐ ┌────▼────┐
   │ Worker  │ │ Worker  │ │ Worker  │
   │ RTX4090 │ │ RTX3080 │ │ T4      │
   │ Score:  │ │ Score:  │ │ Score:  │
   │ 120.5   │ │ 85.2    │ │ 12.0    │
   └─────────┘ └─────────┘ └─────────┘

Job Flow

1. Submit — Client sends a job (Python script + payloads) to the coordinator
2. Queue — Coordinator splits payloads into tasks and stores them in SQLite
3. Claim — Workers pull tasks based on their benchmark score and available memory
4. Execute — Each task runs in an isolated subprocess on the worker
5. Report — Workers report results back to the coordinator
6. Collect — Client polls for results or waits for completion

Benchmark Scoring

Each worker runs a benchmark on join and gets a 0-100 score:

Security

Token Authentication

All communication is authenticated with a shared token:

# Coordinator generates a token
gpumesh serve --token mysecret

# Workers must provide the same token
gpumesh join http://coordinator:8000 --token mysecret

Security Model

Important

Workers execute code from the coordinator. Only share your URL and token with people you trust. gpumesh is designed for trusted networks (home labs, team clusters).

• Code execution: Function tasks (@accelerate) execute in the worker's process
• Plaintext HTTP: All communication uses HTTP. Use Tailscale for encrypted tunnels
• No sandbox: Tasks have full access to the worker machine
• Token secret: Keep it secret. Rotate if compromised

Troubleshooting

Enable Verbose Logging

GPUMESH_VERBOSE=1 gpumesh serve
GPUMESH_VERBOSE=1 gpumesh join http://coordinator:8000 --token mysecret

Force Local-Only Mode

GPUMESH_LOCAL=1 python my_script.py

Limitations

• Python only — Tasks must be Python functions or scripts
• No GPU memory sharing — Each task gets its own process
• No model sharding — Each task runs on one machine at a time
• Single coordinator — Single point of failure (use Tailscale for reliability)
• No built-in encryption — Use Tailscale for encrypted tunnels

Development

Contributing

git clone https://github.com/Samurai007AK/gpumesh.git
cd gpumesh
pip install -e ".[dev]"
pytest

Running Tests

pytest                    # Run all tests
pytest tests/test_api.py  # Run specific test file
pytest -v                 # Verbose output

Building

python -m build
twine check dist/*

License

MIT License. See LICENSE for details.