castkit

castkit is a CLI tool for model quantization and format conversion across GGUF, MLX, GPTQ, AWQ, ONNX, EXL2, and EXL3 workflows, including cross-format conversion via automatic FP16 decast.

Requirements

• Python 3.12+
• Apple Silicon Mac for MLX backend
• NVIDIA GPU + CUDA for GPTQ/AWQ convert
• NVIDIA GPU + CUDA for EXL2/EXL3 convert and measure
• ONNX Runtime for ONNX backend
• llama.cpp (llama-quantize, convert_hf_to_gguf.py) for GGUF convert

Installation

Homebrew (macOS)

brew install schroneko/tap/castkit

pip / uv

uv tool install castkit          # core only
uv tool install castkit[mlx]     # MLX backend (Apple Silicon)
uv tool install castkit[gguf]    # GGUF backend (requires torch)
uv tool install castkit[onnx]    # ONNX backend
uv tool install castkit[exl2]    # EXL2 backend (CUDA required)
uv tool install castkit[exl3]    # EXL3 backend (CUDA required)
uv tool install castkit[all]     # all backends

Quick Start

# version
castkit --version

# convert to GGUF
castkit convert Qwen/Qwen3-0.6B -f gguf -q q4_k_m -o ./output/Qwen3-0.6B.gguf

# convert to MLX 4-bit
castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 -o ./output/Qwen3-0.6B-mlx-4bit

# convert to ONNX
castkit convert Qwen/Qwen3-0.6B -f onnx -o ./output/Qwen3-0.6B-onnx

# convert to EXL2 5bpw
castkit convert Qwen/Qwen3-0.6B -f exl2 -q exl2-5.0 -o ./output/Qwen3-0.6B-exl2

# convert to EXL3 4bpw
castkit convert Qwen/Qwen3-0.6B -f exl3 -q exl3-4.0 -o ./output/Qwen3-0.6B-exl3

# decast (dequantize back to FP16 SafeTensors)
castkit decast ./output/Qwen3-0.6B.gguf -o ./output/Qwen3-0.6B-fp16

# cross-format conversion (GGUF -> GPTQ via automatic FP16 decast)
castkit convert ./output/Qwen3-0.6B.gguf -f gptq -b 4 -o ./output/Qwen3-0.6B-gptq

# model info
castkit info ./output/Qwen3-0.6B.gguf

# perplexity measurement
castkit measure ./output/Qwen3-0.6B.gguf --dataset wikitext-2 --max-samples 128

# importance matrix generation (GGUF)
castkit imatrix ./model -d calibration.txt -o ./output/model.imatrix

Recipes

Define reusable conversion presets in castkit.toml:

[recipes.gguf-standard]
format = "gguf"
quant = "q4_k_m"
imatrix = true
imatrix_data = "calibration.txt"

[recipes.mlx-4bit]
format = "mlx"
bits = 4
group_size = 64

castkit convert Qwen/Qwen3-0.6B --recipe gguf-standard

# batch: convert one model to multiple quants
for q in q4_k_m q5_k_m q6_k q8_0; do
  castkit convert Qwen/Qwen3-0.6B -f gguf -q "$q" -o "./output/Qwen3-0.6B-$q.gguf"
done

Upload to Hugging Face

castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 --upload auto
castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 --upload user/repo-name --public

Supported Formats

License

MIT