quantpilot 1.5.0

QuantPilot tells you which local LLM to run on your PC, in which format and runtime, and why. It sizes every option from the model's own files, checks it against your free memory, ranks the options for the kind of work you do, and measures speed and quality on your machine instead of guessing. Then it makes the file for you.

• Recommends the model, format (GGUF, AWQ, GPTQ, bitsandbytes, NVFP4), runtime (llama.cpp, vLLM, transformers) and context cache for chat, coding, RAG, agents, reasoning, long context, low latency or throughput, with the reasons and how sure it is.
• Sizes exactly: weights and context memory from config.json and the tensor headers (no download of the weights), checked against llama.cpp's own memory numbers on NVIDIA, Apple Silicon and CPU.
• Measures: speed and time to the first token with llama.cpp's benchmark, and quality as perplexity on WikiText-2, compared with the original model. Every result is kept, and the recommendations learn from it.
• Quantizes: one command (or one button) from a Hugging Face model or your own fine-tune to a checked, benchmarked file.

Install

QuantPilot runs with uv on Linux, macOS and Windows, Python 3.10 or newer.

uv tool install quantpilot          # the ocq and quantpilot commands
uvx quantpilot                      # or try it without installing

Each quantization format is an extra, installed when you want it (they bring PyTorch, a large download):

uv tool install "quantpilot[gguf]"                              # GGUF: any PC
uv tool install "quantpilot[gguf,awq,bnb]" --torch-backend auto # NVIDIA: PyTorch for your driver

ocq doctor then says what is ready on your PC and gives the command for anything that is not. The app's This PC page installs the extras with a button. With pip: pip install "quantpilot[gguf]". From the source: git clone https://github.com/glastonvelvarts/QuantPilot && cd QuantPilot && uv sync --extra gguf, then uv run ocq.

Quick start

ocq                                   # the full-screen app: arrow keys and Enter
ocq doctor                            # is everything ready on this PC?
ocq recommend --workload coding       # which model and format to run, and why
ocq feasibility Qwen/Qwen3-8B         # every format of one model on this PC
ocq optimize Qwen/Qwen3-8B -b gguf --gguf-type Q4_K_M --execute
ocq bench ocq-output/Qwen--Qwen3-8B/artifact/Qwen--Qwen3-8B-Q4_K_M.gguf --quality

A model is a Hugging Face id, a folder of your own fine-tune (saved with save_pretrained), a .gguf file, or an Ollama model (ollama:qwen3:8b). ocq models lists the ones already on your PC: Ollama, LM Studio, the Hugging Face cache, your folders.

The app

Run ocq for the full-screen app. Check a model gives the pick with its reasons and confidence, and every format's memory, speed and quality side by side. Quantize makes it with live progress. Benchmark measures any model on the PC and keeps the history. Discover ranks models for a kind of work, Settings keeps a Hugging Face token for gated models.

How it decides

Options that cannot run here are ruled out with the reason. The rest are ranked by the workload's priorities in order (coding: quality, then context, then speed), never by one blended score. Each pick says why it won, what the next options would give and cost, and how sure it is (Low to Very High) with the evidence.

Formats and where they run

Checked against llama.cpp's own numbers on Windows 11 with an RTX 4060 Laptop, Linux (WSL2) with NVIDIA and CPU only, and macOS on Apple Silicon, with GPU and CPU only.

Commands

For scripts and agents

recommend, bench, models, inspect, feasibility, profile, history and doctor take --json, and Python gets the same data:

from ocq import api

pick = api.recommend(workload="coding")["best"]
print(pick["model"]["id"], pick["format"]["label"], pick["quality"]["kept"])

The shape is versioned and only grows within a version: see docs/api.md.

Learn more

• User guide: every command and option, how memory, speed and quality are worked out, fine-tunes on disk, GPUs, mixture of experts, environment variables
• JSON and Python API
• Contributing and changelog

MIT license.