RL-Split / GPUHive

An intelligent reinforcement-learning based runtime abstraction layer for transparent multi-GPU CUDA execution.

Final year project, Department of Computer Science & Engineering, University of Moratuwa. Supervisor: Dr. Adeesha Wijayasiri. Team GPUHive — Kajaluxan S. (220297K), Parishith R. (220444K), Sukithan T. (220625R), Vinushaanth S. (220668B).

The problem, in one paragraph

A machine with several GPUs does not present itself as one larger GPU. CUDA and NCCL provide every mechanism needed to spread one task across several devices, and no policy for deciding how. So programmers pick devices, split ratios, chunk sizes and communication paths by hand — and the right choices depend on the kernel, the input size, which devices are currently busy, and which pairs happen to be well connected. Equal splitting, the practical default, is wrong the moment the devices differ. This project puts a learned policy where that hand-tuning goes.

What it does

                 ┌──────────────────────────────────────────────┐
   application → │  DECISION PLANE      RL agent, live telemetry │
                 └──────────────────┬───────────────────────────┘
                        GATE 1 ─── Security & Feasibility Validator
                 ┌──────────────────┴───────────────────────────┐
                 │  decision-parameter contract  {gpus, shares,  │
                 │   chunk, block, mode, comm_path, config_hash} │
                 └──────────────────┬───────────────────────────┘
                 ┌──────────────────┴───────────────────────────┐
                 │  CODE-GEN PLANE     LLM + hash-sealed cache   │
                 └──────────────────┬───────────────────────────┘
                        GATE 2 ─── Code Security & Integrity Gate
                 ┌──────────────────┴───────────────────────────┐
   result     ←  │  EXECUTION PLANE    CUDA / NCCL, hidden       │
                 └──────────────────────────────────────────────┘

The agent never issues a CUDA call and never writes source. It emits a small declarative object; everything downstream consumes only that object. That separation is the project's architectural claim, and it is what lets every baseline policy in the evaluation drive the same execution path — so a measured difference is attributable to the decision and nothing else.

New here? Read GUIDE.md

GUIDE.md is the complete walkthrough written for someone picking this up cold: what the project is, how a task flows through it, what every file does, every command explained, and how to run it on the university server. Start there.

Try it now — no GPU required

git clone <this repo> && cd gpuhive
make install          # or: . scripts/dev_env.sh   (no install, works in place)

gpuhive doctor                                    # what this machine can do
gpuhive explain --kind matmul --dim M=8192 --dim N=8192 --dim K=8192
gpuhive run --kind reduce --dim N=67108864 --verify
gpuhive security-audit
gpuhive evaluate --repeats 10 --no-optimum

Every command works on a laptop with no GPU, against a simulated machine, and says so in its output. Nothing reports a modelled number as a measurement.

Repository layout

Twelve independently installable packages sharing the gpuhive namespace, plus the CUDA layer and the workload suite. Each package has its own README explaining what it is for and which part of the proposal it implements.

Using it from an application

Python, and the C++ header in cpp/include/gpu_optimizer/agent.h mirrors it field for field:

from gpuhive.runtime import Agent
from gpuhive.core import TaskSpec

agent = Agent("artifacts/policies/rl-split-hetero4/policy.npz")

task = TaskSpec(kind="matmul", dims={"M": 100_000, "N": 100_000, "K": 100_000},
                dtype="fp16", divisible_dim=0, merge="concat")

params = agent.optimize(task)     # decision plane  -> a declarative object
result = agent.execute(task, params)   # execution plane -> the merged result

optimize and execute are separate on purpose. A configuration can be inspected, logged, diffed between policy versions, or replaced by a hand-written one — and the execution layer will run any of them.

Running it on the departmental server

The project targets a server with one GPU today and a second one later, and nothing about that transition requires redoing work.

# 0. One command that does all of the below, and checks it worked:
bash scripts/server_setup.sh

# ...or do it by hand:

# 1. Build the CUDA execution layer (once).
make -C cpp                       # or: make -C cpp NCCL=1
export GPUHIVE_EXEC_LIB=$PWD/cpp/build/libgpuhive_exec.so

# 2. Measure the machine. This is milestone M3 and it is real data.
gpuhive characterise --machine nvml --backend cuda \
    --out artifacts/characterisation.json

# 3. Train. Stages 1 and 2 need no GPU at all, so they can run anywhere;
#    only stage 3 touches the hardware.
gpuhive train --machine uni-server-1gpu --pretrain-steps 500000 \
    --finetune-steps 20000 --finetune-backend cuda

# 4. Evaluate against every baseline, on hardware.
gpuhive evaluate --machine nvml --backend cuda --policy artifacts/policies/…/policy.npz

With one GPU the multi-device actions are masked out, so the framework exercises decision, validation, code generation and execution end to end, and the comparison it can make is single-device against single-device. That is a real but limited result, and the framework labels it as such.

With two GPUs the interesting comparisons open up. The two cards will be identical, so heterogeneity has to be induced — by clock locking or calibrated background load, per Sec. 3.9.1 — and the framework keeps genuine and induced settings separate throughout:

gpuhive evaluate --machine uni-server-2gpu-induced   # clock-locked, reported as induced
gpuhive evaluate --machine hetero-4                  # simulated genuine heterogeneity

Note honestly what induced heterogeneity cannot show: clock locking scales arithmetic and memory throughput together, so it does not change which kernel each device is relatively good at. The kernel-dependent device ordering that motivates the whole project (Sec. 1.2) is only observable on genuinely different cards.

What is honest about the numbers

This matters more than any individual result, so it is stated plainly:

• Simulated is labelled simulated. Every ExecutionMetrics carries a backend field; every results table prints whether it was MEASURED on hardware or PREDICTED by the cost model; the Markdown report puts a warning block at the top of a predicted run.
• Measured interconnect is distinguished from estimated. TopologyProbe carries measured=True/False and the two are never pooled.
• The cost model reports its own error. gpuhive calibrate prints median APE, p90 APE and rank correlation, and warns when the error is above 30%.
• The targets can fail. gpuhive evaluate checks Sec. 3.8's falsifiable targets and prints PASS or FAIL with the measured value either way.
• The security gates are measured against benign controls too. A gate that rejects everything scores 100% coverage and is useless, so the audit reports false positives and cases caught by the wrong check.

Development

make test          # the full suite
make lint          # syntax and import checks across every package
make audit         # security validation coverage
make demo          # train briefly, then evaluate, end to end
make cpp           # build the CUDA execution layer
make cpp-stub      # build the CPU ABI stub (no CUDA needed)

The test suite runs in seconds and needs no GPU. It includes gradient checks against finite differences for every autodiff operation and every action distribution, because hand-written backward passes are exactly the kind of code that is wrong in a way ordinary tests do not notice.

Documentation

• docs/architecture.md — how the planes fit together and why
• docs/proposal-map.md — every section, objective and gap of the proposal, mapped to code
• docs/deployment.md — the one-GPU to two-GPU path, in detail
• docs/experiments.md — how to reproduce every result in the evaluation plan
• docs/findings.md — what building it revealed that the proposal did not anticipate; several of these are results

Status

Implemented end to end and running: the contract, both security gates, the cost model and its calibration, all three execution backends, the CUDA/NCCL layer, the code-generation plane with four LLM providers and a deterministic fallback, the Gymnasium environment, masked PPO with the Dirichlet allocation head, all seven baselines, the runtime, and the evaluation harness with ablations and transfer studies.

Not yet done, and requiring hardware: measured characterisation, cost-model calibration against real timings, hardware fine-tuning, and every number in the evaluation reported as measured rather than modelled.