vllm-omni-humming

Plugin that lets stock vLLM-Omni serve pre-quantized humming Qwen-Image checkpoints.

Install it into the vLLM-Omni environment and serve normally — the plugin registers itself through vLLM-Omni's standard plugin entry point, and quantized checkpoints are detected automatically from their transformer/config.json. No flags, no environment variables, no configuration.

Compatibility

Linux only, NVIDIA GPUs (tested on 30-, 40-, 50-series and RTX Pro 6000).

humming-kernels >= 0.1.11 is required (older versions produce corrupted images on some GPUs). Install it explicitly as shown below, or via this package's [kernels] extra.

What it patches

Five small, humming-only runtime patches against stock vLLM-Omni; other quantization methods are untouched. Each is a separate source module documenting the exact upstream symbol it targets:

• _preimport.py — import the real humming package before vLLM-Omni's stub.
• _factory.py — register humming as a quantization method for checkpoints with a quantization_config.
• _loader.py — load packed weights on CPU and repack per module on GPU (keeps the load peak low enough for 32 GiB cards), and accept humming's tensor-name suffixes.
• _modpin.py — route quantized modulation layers (img_mod.1/txt_mod.1), which stock vLLM-Omni keeps in bf16.

Install

Linux only, NVIDIA GPUs (tested on 30-, 40-, 50-series and RTX Pro 6000).

# 0) install conda (skip if you already have conda)
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p ~/miniforge3 && source ~/miniforge3/bin/activate

# 1) conda env
conda create -y -n qwen-image-humming python=3.12
conda activate qwen-image-humming

# 2) torch trio (cu129)
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
  --index-url https://download.pytorch.org/whl/cu129

# 3) vLLM 0.24.0 cu129 wheel
printf 'torch==2.11.0+cu129\ntorchvision==0.26.0+cu129\ntorchaudio==2.11.0+cu129\n' > constraints.txt
PIP_CONSTRAINT=constraints.txt pip install \
  https://github.com/vllm-project/vllm/releases/download/v0.24.0/vllm-0.24.0+cu129-cp38-abi3-manylinux_2_28_x86_64.whl

# 4) vLLM-Omni 0.24.0
PIP_CONSTRAINT=constraints.txt pip install vllm-omni==0.24.0

# 5) this plugin
PIP_CONSTRAINT=constraints.txt pip install vllm-omni-humming==0.3.0

# 6) humming kernels
pip install --no-deps "humming-kernels>=0.1.11"

Requirements: NVIDIA GPU (SM75+), driver ≥ 575, gcc. No CUDA toolkit needed — kernels are JIT-compiled on first serve (a few minutes once, cached afterwards). Any Python 3.10–3.13 environment works in place of conda (a system venv additionally needs the python3-venv and python3-dev packages).

Serve

vllm-omni serve /path/to/checkpoint \
  --omni --served-model-name Qwen/Qwen-Image-2512 \
  --enable-cpu-offload \
  --port 8124

Keep --enable-cpu-offload on GPUs with ≤ 32 GiB; drop it on larger cards for full speed. Generate an image (OpenAI images API):

curl -s http://127.0.0.1:8124/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen-Image-2512",
    "prompt": "A red panda on a mossy log at dawn",
    "size": "1024x1024",
    "num_inference_steps": 20,
    "true_cfg_scale": 2.5,
    "negative_prompt": "blurry, low quality, distorted, watermark",
    "seed": 1234
  }' \
  | python3 -c "import sys,json,base64; d=json.load(sys.stdin); open('out.png','wb').write(base64.b64decode(d['data'][0]['b64_json']))"

Always send a negative_prompt — vLLM-Omni disables classifier-free guidance without one.

Troubleshooting

• TypeError: HummingConfig.__init__() got an unexpected keyword argument ... — the plugin isn't installed in the serving venv (pip show vllm-omni-humming).
• Shape/dtype mismatch on img_mod.1 / txt_mod.1 at load — same cause.
• Corrupted images — humming-kernels too old; pip install --no-deps "humming-kernels>=0.1.11".

License

Apache-2.0. See LICENSE.