gguf-cpp

One package for working with GGUF models locally: an OpenAI-compatible LLM server, a diffusion image/video/audio generator and a GGUF metadata/tensor editor with a built-in quantizer — three panels on one GUI, powered by one unified gguf.cpp engine compiled in a single build with a shared set of ggml kernels.

Install

pip install gguf-cpp

The build compiles the bundled engine (CPU by default, Metal on macOS). GPU backends are opt-in at install time:

GGUF_CPP_CUDA=1 pip install gguf-cpp     # NVIDIA
GGUF_CPP_HIP=1 pip install gguf-cpp      # AMD ROCm
GGUF_CPP_VULKAN=1 pip install gguf-cpp   # Vulkan

Run

gguf-cpp                 # unified GUI — Server / Diffuser / Editor panels
python -m gguf_cpp       # same thing

Each panel also runs on its own, exactly like the standalone gguf-server / gguf-diffusion / gguf-editor packages did:

gguf-cpp server          # LLM server GUI
gguf-cpp diffuser        # image generation GUI
gguf-cpp editor          # GGUF editor GUI

And the engines are directly scriptable from the CLI:

gguf-cpp server engine -- --model model.gguf --port 8888
gguf-cpp diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
gguf-cpp editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
gguf-cpp editor devices

Layout

vendor/engine/           the unified gguf.cpp engine (one CMake build)
  kernels/               shared ggml kernels (CPU + optional GPU backends)
  src/ common/ mtmd/     GGUF LLM runtime
  app/                   the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library (its own quant kernels)
src/gguf_cpp/            the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell