castkit 0.1.1

Universal model quantization and format conversion CLI

castkit

castkit is a CLI tool for model quantization and format conversion across GGUF, MLX, GPTQ, AWQ, ONNX, EXL2, and EXL3 workflows, including cross-format conversion via automatic FP16 decast.

Requirements

• Python 3.12+
• Apple Silicon Mac for MLX backend
• NVIDIA GPU + CUDA for GPTQ/AWQ convert
• NVIDIA GPU + CUDA for EXL2/EXL3 convert and measure
• ONNX Runtime for ONNX backend
• llama.cpp (llama-quantize, convert_hf_to_gguf.py) for GGUF convert

Installation

Homebrew (macOS)

brew install schroneko/tap/castkit

pip / uv

uv tool install castkit          # core only
uv tool install castkit[mlx]     # MLX backend (Apple Silicon)
uv tool install castkit[gguf]    # GGUF backend (requires torch)
uv tool install castkit[onnx]    # ONNX backend
uv tool install castkit[exl2]    # EXL2 backend (CUDA required)
uv tool install castkit[exl3]    # EXL3 backend (CUDA required)
uv tool install castkit[all]     # all backends

Quick Start

# version
castkit --version

# convert to GGUF
castkit convert Qwen/Qwen3-0.6B -f gguf -q q4_k_m -o ./output/Qwen3-0.6B.gguf

# convert to MLX 4-bit
castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 -o ./output/Qwen3-0.6B-mlx-4bit

# convert to ONNX
castkit convert Qwen/Qwen3-0.6B -f onnx -o ./output/Qwen3-0.6B-onnx

# convert to EXL2 5bpw
castkit convert Qwen/Qwen3-0.6B -f exl2 -q exl2-5.0 -o ./output/Qwen3-0.6B-exl2

# convert to EXL3 4bpw
castkit convert Qwen/Qwen3-0.6B -f exl3 -q exl3-4.0 -o ./output/Qwen3-0.6B-exl3

# decast (dequantize back to FP16 SafeTensors)
castkit decast ./output/Qwen3-0.6B.gguf -o ./output/Qwen3-0.6B-fp16

# cross-format conversion (GGUF -> GPTQ via automatic FP16 decast)
castkit convert ./output/Qwen3-0.6B.gguf -f gptq -b 4 -o ./output/Qwen3-0.6B-gptq

# model info
castkit info ./output/Qwen3-0.6B.gguf

# perplexity measurement
castkit measure ./output/Qwen3-0.6B.gguf --dataset wikitext-2 --max-samples 128

# importance matrix generation (GGUF)
castkit imatrix ./model -d calibration.txt -o ./output/model.imatrix

Recipes

Define reusable conversion presets in castkit.toml:

[recipes.gguf-standard]
format = "gguf"
quant = "q4_k_m"
imatrix = true
imatrix_data = "calibration.txt"

[recipes.mlx-4bit]
format = "mlx"
bits = 4
group_size = 64

castkit convert Qwen/Qwen3-0.6B --recipe gguf-standard

# batch: convert one model to multiple quants
for q in q4_k_m q5_k_m q6_k q8_0; do
  castkit convert Qwen/Qwen3-0.6B -f gguf -q "$q" -o "./output/Qwen3-0.6B-$q.gguf"
done

Upload to Hugging Face

castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 --upload auto
castkit convert Qwen/Qwen3-0.6B -f mlx -b 4 --upload user/repo-name --public

Supported Formats

Format	Convert	Decast	Info	Measure
GGUF	Yes	Yes	Yes	Yes
MLX	Yes	Yes	Yes	Yes
GPTQ	Yes (CUDA)	Yes	Yes	Yes
AWQ	Yes (CUDA)	Yes	Yes	Yes
ONNX	Yes	Yes	Yes	Yes
EXL2	Yes (CUDA)	Yes	Yes	Yes (CUDA)
EXL3	Yes (CUDA)	Yes	Yes	Yes (CUDA)
FP16	Yes	Yes	Yes	Yes
BF16	Yes	Yes	Yes	Yes
FP32	Yes	Yes	Yes	Yes

License

MIT