ggk

One package for working with GGUF models locally: an OpenAI-compatible LLM server, a diffusion image/video/audio generator and a GGUF metadata/tensor editor with a built-in quantizer — three panels on one GUI, powered by one unified engine compiled in a single build on top of gk, an independent tensor library. There is no ggml anywhere in the tree.

Install

pip install ggk

The build compiles the bundled engine (CPU by default, Metal on macOS). GPU backends are opt-in at install time:

GGK_CUDA=1 pip install ggk     # NVIDIA
GGK_HIP=1 pip install ggk      # AMD ROCm
GGK_VULKAN=1 pip install ggk   # Vulkan

Each switch drives the whole engine — the server, the diffusion runtime and the multimodal projectors all evaluate their graphs on the one gk build.

Building a wheel

python -m build --wheel

--wheel is not optional for a GPU build. Plain python -m build builds an sdist first and then compiles the wheel from it in a temporary directory, so every run starts from scratch and a multi-hour CUDA build spends those hours inside a directory the OS is free to sweep — on Windows that surfaces at the very end as FileNotFoundError: ...\wheel\scripts from scikit-build-core's packaging step, long after the compile succeeded. With --wheel the CMake build directory stays at build/{wheel_tag} in the tree and rebuilds are incremental.

On Windows a CUDA build needs MSVC, because that is the only host compiler nvcc accepts there — run it from a vcvars64 shell. Use Ninja rather than -G "Visual Studio 17 2022": scikit-build-core passes no -j on the pyproject path and CMake's Visual Studio generator does not set /MP, so an MSBuild-driven build compiles one file at a time.

call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
set CMAKE_ARGS=-G Ninja -DGGK_CUDA=ON
python -m build --wheel

The build works its own architecture list out from nvcc and the installed GPUs. A wheel built on one machine for another should say what it targets: -DGK_CUDA_ARCHITECTURES="75;86;89;120".

Run

ggk                 # unified GUI — Server / Diffuser / Editor panels
python -m ggk       # same thing

Each panel also runs on its own, exactly like the standalone gguf-server / gguf-diffusion / gguf-editor packages did:

ggk server          # LLM server GUI
ggk diffuser        # image generation GUI
ggk editor          # GGUF editor GUI

And the engines are directly scriptable from the CLI:

ggk server engine -- --model model.gguf --port 8888
ggk diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
ggk editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
ggk editor devices

Any other leading option is a diffusion engine command line as is:

ggk --diffusion-model flux3-image-klein-nvfp4.gguf --vae pig_flux2_vae_fp32-f16.gguf \
    --llm qwen3vl-4b-it-q4_k_m.gguf -r sheep.png -p "a sheep in sunglasses" \
    --cfg-scale 1 --steps 4 --sampling-method euler --diffusion-fa -o out.png

FLUX.3 prompt upsampler

FLUX.3 Image Klein is trained on the captions its prompt upsampler writes (a Qwen3.5-9B VLM); the text encoder is Qwen3-VL-4B. The upsampler runs on the bundled LLM server engine before the diffusion engine starts, with Black Forest Labs' external-VLM system prompts (src/ggk/diffuser/pup_prompts_external_vlm.json):

ggk --diffusion-model flux3-image-klein-nvfp4.gguf --vae pig_flux2_vae_fp32-f16.gguf \
    --llm qwen3vl-4b-it-q4_k_m.gguf --prompt-upsampler qwen3.5-9b-q4_k_m.gguf \
    -r sheep.png -p "a sheep in sunglasses" --cfg-scale 1 --steps 4 -o out.png

--llm qwen3.5-9b-q4_k_m.gguf on a FLUX.3 run means the same (the Qwen3-VL-4B GGUF next to the diffusion model is used as the text encoder). Reference images go to the upsampler through its mmproj-*.gguf, found by name next to the model or given with --prompt-upsampler-vision. --prompt-upsampler-out caption.txt keeps the caption, --no-prompt-upsampler sends the prompt as typed, and python -m ggk.diffuser.pup --model qwen3.5-9b-q4_k_m.gguf -p "..." upsamples alone.

MOSS-Image (Base / SFT)

MOSS-Image is a 5B single-stream DiT on the FLUX.2 VAE whose text encoder is Qwen3.5-9B itself (last hidden state, chat template with an open <think> block). Pass the stock llama.cpp Qwen3.5 GGUF as --llm; its 248k-token tokenizer is read from the GGUF's metadata, no tokenizer file needed. The upstream schedule is a flow shift of 5 (--flow-shift overrides), 1024 px is native and dimensions must be multiples of 16; a CFG run skips DiT layer 22 on the negative branch like the reference pipeline.

ggk --diffusion-model moss-image-sft-q4_k_s.gguf --vae pig_flux2_vae_fp32-f16.gguf \
    --llm qwen3.5-9b-q4_k_m.gguf -p "a lovely cat holding a sign that says moss" \
    --cfg-scale 3 --steps 30 -o out.png

The DiT has no reference-latent path. With --llm_vision mmproj-qwen3.5-9b-f16.gguf a -r reference image goes through Qwen3.5's vision tower into the user turn (before the prompt, with Qwen3-VL M-RoPE positions), so the text features are image-conditioned; it never enters the VAE or the DiT:

ggk --diffusion-model moss-image-sft-q4_k_s.gguf --vae pig_flux2_vae_fp32-f16.gguf \
    --llm qwen3.5-9b-q4_k_m.gguf --llm_vision mmproj-qwen3.5-9b-f16.gguf \
    -r sheep.png -p "a sheep in sunglasses" --cfg-scale 1 --steps 4 --diffusion-fa -o out.png

Reference images are resized (Qwen3-VL smart_resize, multiples of 32) to at most 1024x1024 pixels, i.e. 1024 vision tokens.

Integer weight types (i8 / i16 / i32 / i64)

The plain integer tensor types carry no scale of their own, so a weight cast straight to i8 would be all zeros. When the quantizer converts a tensor to an integer type (--type i8, or a rule like embed_tokens.weight=i8) it stores a symmetric per-row scale in the file's metadata,

int_scale.<tensor name> = [f32; rows]      # value = int * scale[row]

and the diffusion engine folds the scale back in at load time (i8 computes as q8_0, i16 as f16, i32/i64 as f32 on every backend). The LLM server does the same conversion while it loads (which turns mmap off for that model, since the converted weights need their own buffers). Converting the tensor to any other type drops the key again. An integer tensor without the key is what it always was in GGUF: raw integers (token tables, indices).

The editor marks such tensors with a ×scale badge, and every tensor row has a Values button that decodes a slice of one row through the quantizer library (scales applied, with a "raw integers" toggle for scaled weights). Renaming, deleting or merging a scaled tensor keeps its int_scale.* entry in step on save.

Which hardware it picked

Every engine prints the device list to stderr before it does anything else:

gk: found 2 devices
  CUDA0: NVIDIA GeForce RTX 4050 Laptop GPU, 6140 MiB | compute capability = 8.9 | SMs = 20 | shared memory = 99 KiB | built for = 89
  CPU: gk CPU backend, 32014 MiB | SIMD = AVX2 | AVX2 = 1 | FMA = 1 | F16C = 1 | F16_VEC = 1

If a GPU you expected is missing, the install was a CPU-only one (the GPU switches above are opt-in at install time) or its driver was not found — either way the run works, on the CPU, at CPU speed, which is otherwise indistinguishable from a slow GPU. GK_QUIET=1 suppresses the banner.

Layout

vendor/engine/           the unified ggk engine (one CMake build)
  gk/                    the gk compute kernels (CPU + optional GPU backends)
  gk/compat/             the historical ggml C API, implemented on gk
  src/ common/ mtmd/     GGUF LLM runtime
  app/                   the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library (its own quant kernels)
src/ggk/                 the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell

Nothing above gk/compat/ knows gk exists: the runtimes include the same ggml.h / ggml-backend.h / gguf.h headers and call the same functions they always did, while graph building, allocation, scheduling and the kernels themselves are gk's. See vendor/engine/README.md for the engine's own build options.

The editor's quantizer stays independent — its qz_* codec is compiled both into the quantizer library the editor drives and into gk itself, so the encoder and the runtimes' decoder can never disagree about a GGUF block.

Metadata

Release files for ggk 0.9.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ggk 0.9.0
File	Size	Uploaded	Provenance
ggk-0.9.0.tar.gz	36.4 MB	Oct 18, 2026	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for ggk 0.9.0
File	Interpreter	ABI	Platform	Reset
ggk-0.9.0-py3-none-win_amd64.whl 87.2 MB Oct 18, 2026 Without provenance	Python 3	none	Windows x86-64	Details

Total release size: 123.6 MB

Release files / ggk-0.9.0.tar.gz

Download URL	ggk-0.9.0.tar.gz
Size	36.4 MB
Tags	Source
SHA-256 checksum How to use checksums	`b5e82f67a06076f9439d78d5683f7666c54d95a63382e6bc1bfdbf4e8c67a8ad`
BLAKE2b-256 checksum How to use checksums	`3611879bc5db86822a7a3baae3e8a8c63e2bdb12309dd13594c25647ffaea97c`
Upload date	Oct 18, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.11.9`

Release files / ggk-0.9.0-py3-none-win_amd64.whl

Download URL	ggk-0.9.0-py3-none-win_amd64.whl
Size	87.2 MB
Tags	Python 3 Windows x86-64
SHA-256 checksum How to use checksums	`49ed2ee8a98290aee49d78cff47dd49307d2222293f4b5ea30aa8ec254e6d797`
BLAKE2b-256 checksum How to use checksums	`6fb7b4a363e6c4744b97f197ad5b64062e0743faf2c80890f5642f5067d4c515`
Upload date	Oct 18, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.11.9`

Security for ggk 0.9.0

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.