gpuctl-cli 0.9.0

AI Computing Power Scheduling Platform

🚀 Schedule GPU Clusters Like Writing Python Scripts

Declarative YAML · Zero K8s Knowledge · Resource Pool Isolation

简体中文 • Quick Start • Documentation • Features

---

✨ Why gpuctl

One Command to Rule Them All gpuctl create -f job.yaml Say goodbye to 100+ lines of K8s YAML, submit tasks with declarative configuration

Multi-Team Resource Isolation Training Pool / Inference Pool / Dev Pool Logical isolation prevents resource contention, with quota management per team

Distributed Training Built-In Indexed Job + Headless Service Set resources.nodes: N — platform auto-injects DDP env vars (MASTER_ADDR / RANK / WORLD_SIZE)

One-Stop Monitoring Logs / Events / Resource Usage No more kubectl get pods to find pod names

ML Engineer Friendly kind / job / resources Familiar YAML syntax, no need to understand Pod/Deployment concepts

Namespace-Level Quotas CPU / Memory / GPU Auto-bind ResourceQuota when creating Namespace

Complete API Support HTTP / WebSocket Easy integration with MLOps platforms or third-party tools

Existing K8s Cluster Ready to Use No cluster configuration changes, no impact on existing workloads

Zero-config NFS storage on every job Operator runs gpuctl init once → every job auto-mounts a persistent, per-user /home/jovyan (read-write) and a shared /datasets (read-only). No mount paths, no storage classes, no PVCs in user YAML. Files survive restarts and are shared across a user's Notebook and Training jobs.

---

🚀 Quick Start

# 1. Install CLI
pip install gpuctl

# (operator, once) Enable transparent persistent storage for every job
gpuctl init --nfs-server <IP> --nfs-path /exports

# 2. Submit LLM fine-tuning task (4x A100)
cat > training.yaml << 'EOF'
kind: training
version: v0.1
job:
  name: qwen2-7b-sft
environment:
  image: llama-factory:latest
  command: ["llamafactory-cli", "train", "--stage", "sft"]
resources:
  pool: training-pool
  gpu: 4
  cpu: 32
  memory: 128Gi
EOF

gpuctl create -f training.yaml

# 3. Check task status
gpuctl get jobs

# 4. View logs in real-time
gpuctl logs qwen2-7b-sft -f

---

🆚 gpuctl vs Native Kubectl

Scenario	✨ gpuctl Way	Native Kubectl Way
📝 Submit Training Task	Just 15-20 lines of declarative config, fill in familiar fields like kind, job.name, resources.gpu, and submit	Write 120+ lines of K8s YAML, manually create Secret, ConfigMap, Job resources, understand PodSpec, ResourceRequirements, VolumeMounts
📊 Check Task Status	One command for all tasks gpuctl get jobs, auto-aggregate Pod status, show task name, status, resource usage	kubectl get jobs to find Job, then get pods -l job-name=xxx to find Pod, finally describe pod for details, tedious process
🔍 View Task Logs	Use task name directly gpuctl logs <job-name> -f, auto-track Pod changes, support multi-replica aggregated logs	Remember Pod name (e.g. training-job-7d9f4b8c5-x2mnp), run kubectl logs <pod-name> -f, re-find after Pod restart
🧠 Multi-Node Distributed Training	Just set resources.nodes: N, platform creates an Indexed Job + Headless Service and auto-injects DDP rendezvous env vars (MASTER_ADDR, MASTER_PORT, WORLD_SIZE, RANK, LOCAL_RANK); all workers share one NFS /home/jovyan for checkpoints	Manually create an Indexed/JobSet + Headless Service, wire up MASTER_ADDR/RANK/WORLD_SIZE, provision shared storage, understand GPU communication and process groups
🏊 Resource Pool Management	Declarative pool config, pool: training-pool auto-schedules to corresponding node group, supports multi-team isolation and quota control	Manually bind nodes via LabelSelector and NodeAffinity, maintain complex scheduling strategies and resource limits per team
📋 Resource Quota Management	Quota auto-created with Namespace, gpuctl describe quota one-click view of used/total, auto-reject with friendly message when exceeded	Manually create ResourceQuota and LimitRange, configure per Namespace, query usage multiple times for aggregation
⚡ Deploy Inference Service	Auto-create Deployment + Service, declare replicas and port, auto-generate NodePort to expose service, built-in readiness probe	Create Deployment, Service, Ingress/NodePort separately, configure HPA auto-scaling, understand Service types and network policies
📓 Launch Notebook	One-click JupyterLab launch, auto-generate access link, support custom images and passwords, auto-mount storage volumes	Manually create StatefulSet, Headless Service, Ingress, configure PVC storage, handle Jupyter Token and passwords

---

🏗️ Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────────────────────┐
│   User      │────▶│  gpuctl CLI │────▶│  K8s Job/Deployment/        │
│  (YAML)     │     │   / REST API│     │  StatefulSet + Service      │
└─────────────┘     └─────────────┘     └─────────────────────────────┘

---

📚 Documentation

Complete documentation is available in the docs/ directory, or check out the quick navigation below:

Getting Started

• Quick Start — Get started with gpuctl in 5 minutes
• Installation Guide — Detailed installation steps

User Guides

• Training Tasks — LLM fine-tuning, conda env reuse, multi-node distributed training
• Persistent Storage — transparent NFS /home/jovyan + /datasets, zero config in job YAML
• Inference Services — VLLM inference deployment; single-node tensor-parallel and multi-node (resources.nodes: N) model-parallel serving
• Notebooks — JupyterLab interactive development
• Resource Pool Management — GPU resource pool configuration

Reference

• CLI Commands — Complete command reference
• API Documentation — RESTful API specifications
• FAQ — Frequently asked questions and troubleshooting

Development & Contribution

• Architecture Design — System design documentation
• Local Development — Development environment setup
• Contributing Guide — How to contribute

---

💻 Installation

Prerequisites

• Python 3.8+
• Kubernetes cluster access (via kubectl)

From PyPI (Recommended)

pip install gpuctl

From Source

git clone https://github.com/runwhere-ai/gpuctl.git
cd gpuctl
pip install -e .

Binary Download

# Linux
wget https://github.com/runwhere-ai/gpuctl/releases/latest/download/gpuctl-linux-amd64
chmod +x gpuctl-linux-amd64
sudo mv gpuctl-linux-amd64 /usr/local/bin/gpuctl

---

🌟 Show Your Support

If gpuctl helps you, please give us a ⭐️ Star!

---

📄 License

MIT License © 2024 GPU Control Team