vramsuite 0.5.0

Native GPU memory diagnostics for local AI.
Know whether your workflow fits in VRAM before it crashes with out-of-memory.

VRAM Suite talks to NVIDIA libraries directly (NVML, CUDA driver and runtime, cuBLAS, cuBLASLt) and shows what is really happening with GPU memory: what the card is, who holds VRAM, how big a model is, and how fast weights move over PCIe.

Install

pip install vramsuite
vramsuite doctor

Only the NVIDIA driver is needed: no PyTorch, no CUDA Toolkit. The cuBLAS / cuBLASLt compute checks (doctor --compute-check) need cuBLAS, from the CUDA Toolkit or from NVIDIA's wheels:

pip install "vramsuite[compute]"

No Python? Download vramsuite.exe (Windows) from Releases and run vramsuite.exe doctor in a terminal, or double-click it.

From source:

git clone https://github.com/k1n0F/vramsuite
cd vramsuite
uv sync
uv run vramsuite doctor

Commands

Every command supports --json. Full documentation: docs/.

Why native

• No PyTorch, no Triton. A diagnostic tool should not load gigabytes of framework and its own CUDA libraries just to read how much memory is free. PyTorch is only detected from package metadata.
• No DLL conflicts. Importing torch makes Windows bind cuBLAS to torch's copy of cuBLASLt by name. VRAM Suite never imports it, so its numbers are not affected by a second CUDA stack.
• Observes from outside. ComfyUI, games and browsers are measured through OS and driver counters; nothing has to run inside them.
• Explicit and bounded. Nothing heavy runs by default, every allocation is released, and a value that cannot be read is reported with a reason instead of disappearing.

More: Native backends.

See it work

Who holds VRAM right now — vramsuite processes

How big a model really is — vramsuite inspect on a WAN 2.2 14B NVFP4 checkpoint: FP8 and packed FP4 weights, read from the header in milliseconds

What offloading costs — vramsuite transfer

Real numbers

Measured on an RTX 5080 in a slot that runs at PCIe Gen5 x8:

VRAM is about 30 times faster than the link to system RAM. That gap is why a model that does not fit in VRAM slows down so much when its weights are offloaded, and why VRAM Suite measures both.

Architecture

flowchart LR
    CLI["CLI<br/>vramsuite ..."] --> CORE
    API["Python API<br/>import vramsuite"] --> CORE

    subgraph CORE["core"]
        direction TB
        doctor["doctor / vramcard"]
        inspect["model_inspect"]
        processes["processes"]
        transfer["transfer"]
        estimate["estimate"]
        workflow["workflow (ComfyUI)"]
        probe["probe / risk"]
    end

    subgraph BACKENDS["backends (ctypes)"]
        direction TB
        nvml["nvml"]
        driver["cuda driver"]
        cuda["cuda runtime"]
        blas["cublas / cublaslt"]
        win["windows_gpu"]
    end

    CORE --> BACKENDS
    nvml --> NV["nvml.dll / libnvidia-ml.so"]
    driver --> DRV["nvcuda.dll / libcuda.so"]
    cuda --> RT["cudart"]
    blas --> BL["cuBLAS / cuBLASLt"]
    win --> OS["pdh / gdi32 / ntdll"]

vramsuite/
  __init__.py          Public Python API
  cli/main.py          CLI entrypoint
  core/
    doctor.py          Structured diagnostics API (+ native compute checks)
    fingerprint.py     Runtime, NVML, CUDA runtime, cuBLAS, torch-metadata fingerprint
    vramcard.py        .vramcard creation, GPU matching (NVML <-> CUDA by PCI)
    probe.py           Bounded allocation probe (cuMemAlloc)
    risk.py            Basic OOM risk estimation
    model_inspect.py   .safetensors header reader (exact weight sizes)
    processes.py       Per-process VRAM usage and peaks
    transfer.py        Host <-> device bandwidth benchmark
    estimate.py        Workflow memory estimate and offloading cost
    workflow.py        ComfyUI workflow JSON -> models, settings, hashes, warnings -> estimate
    profile.py         Passive ComfyUI run capture: peak VRAM + run time per config hash
    advise.py          Suggestions with numbers for a workflow estimate
    comfy.py           Workflow file -> analysis + profile + estimate + suggestions
    bundle.py          Anonymized zip of card, validation and profiles
    compare.py         .vramcard comparison
    reports.py         Rich terminal report rendering
    runtime.py         OS/Python/WSL/container detection
  backends/
    native_loader.py   DLL/.so discovery (CUDA Toolkit first, venv fallback)
    nvml.py            NVML: memory, device details, PCIe, clocks, power, processes
    cuda.py            CUDA runtime: memory, device attributes, malloc/memcpy, pinned memory
    cuda_driver.py     CUDA driver API: probe and transfers with only the NVIDIA driver installed
    cublas.py          cuBLAS handle and SGEMM
    cublaslt/          cuBLASLt descriptors, heuristics, matmul
    windows_gpu.py     Windows PDH GPU counters, LUID -> PCI, process info
  compute/
    compute_math.py    cuBLAS SGEMM sanity check
    cublaslt_math.py   cuBLASLt FP32 / FP8 GEMM checks
    minifloat.py       Pure-Python FP8 / FP4 encode/decode

Python API

import vramsuite

result = vramsuite.run_doctor(with_compute_check=True)
print(result["gpu"])
print(result["memory"])

card = vramsuite.create_vramcard()
vramsuite.save_vramcard("system.vramcard.json", card=card)

from vramsuite.core.model_inspect import inspect_model
from vramsuite.core.processes import watch_gpu_processes
from vramsuite.core.transfer import run_transfer_benchmark

print(inspect_model("model.safetensors")["total_mb"])
print(watch_gpu_processes(seconds=30)["processes"][:3])
print(run_transfer_benchmark()["results"])

Roadmap

• Native CUDA runtime, cuBLAS and cuBLASLt backends (no PyTorch)
• CUDA driver API backend: probe and transfers without the CUDA Toolkit
• Device profile in .vramcard, NVML <-> CUDA matching by PCI address
• Compute check layer (cuBLAS / cuBLASLt FP32, FP8)
• Model file inspection from .safetensors headers
• Per-process VRAM tracking (Windows PDH / Linux NVML)
• Host <-> device transfer benchmark
• Workflow memory estimation: weights + activations + offloading cost vs the device profile
• ComfyUI workflow analysis
• Profile validation and schema checks
• Offline workflow analysis: settings, graph / config hashes, risky patterns (v0.2)
• Workflow profiles: real peak VRAM and run time from ComfyUI, used by estimate (v0.3)
• Suggestions with numbers: FP8 variants on disk, largest size that fits, offload and reload cost (v0.4)
• PyPI package with a [compute] extra, Windows exe on GitHub Releases, anonymized bundle (v0.5)

Out of scope (separate kernel project): custom attention kernels, FP4 block-scaled GEMM, kernel autotuning.

Tested on

Other NVIDIA GPUs should work; FP8 compute checks need Ada (sm_89) or newer.

Status

Alpha (v0.5). The CLI, .vramcard format and Python API can still change between versions.

License

MIT © k1n0F