CanIRunLLM

Find out which open-source LLMs your machine can actually run — and which one you should pick — with an automatic hardware scan and a local dashboard.

CanIRunLLM inspects your real CPU, RAM, and GPU/VRAM, checks that against a registry of real open-weight models (Qwen, Llama, Mistral, Gemma, Phi, DeepSeek, and more), and gives you a plain-language verdict — not just a raw "fits/doesn't fit" table. It runs entirely on your machine: no account, no cloud calls, no telemetry.

$ canirunllm scan

CanIRunLLM

Checking your computer...
  CPU detected
  RAM detected
  GPU detected
  VRAM detected

Analyzing local AI models...
  62 models analyzed

You can run 14 model(s) comfortably.
7 more can run with CPU/RAM offload (slower).

Dashboard:
  http://127.0.0.1:8765

Opening browser...

The browser dashboard shows a friendly "what can I run / what should I run" report — including estimated throughput and comparison charts — with the full technical breakdown (VRAM, RAM, KV cache, quantization, runtime, confidence) available behind a "Technical details" toggle for anyone who wants it. Clicking a model shows real, verified commands for running it yourself (llama.cpp, Ollama) — and for a growing set of models, a Download & Run button that does it for you, right there in the dashboard, with a live chat box to try it once it's up.

Table of contents

• Why this exists
• Install
• Quick start
• What it actually checks
• Download & run models directly
• Architecture
• Current limitations
• Contributing
• Testing
• License

Why this exists

Open-weight models are closing the gap on proprietary ones fast, and running them locally means no API keys, no per-token bills, no vendor lock-in, and your data never leaving your machine. That future is arriving whether or not any single vendor cooperates.

But "just run it locally" quietly assumes you already know which model fits your hardware, which quantization to use, whether it needs multi-GPU or CPU/RAM offloading, and which runtime actually supports that combination — and gets it wrong just as quietly, with an OOM crash or a model that technically loads but is unusably slow.

CanIRunLLM exists to answer the question that actually matters before any of that: given the machine you actually have, what should you run? Not "is this hardware capable in theory," but "here's what you can run today, here's what you should pick, and here's why" — so local, open-weight models become something you can act on with confidence, not something you find out by trial and (expensive) error.

Install

Windows, no Python needed: download CanIRunLLM.exe from the latest release (or from canirunllm.vercel.app) and double-click it.

Any OS with Python already installed:

pip install canirunllm
canirunllm scan

Run canirunllm --help any time to see every command.

py -0

py -3.11 -m pip install canirunllm
py -3.11 -m canirunllm.cli scan

or from source:

git clone https://github.com/jashwanthsai678/CaniRunLLM.git
cd CaniRunLLM
pip install -e .

Quick start

canirunllm scan               # scan hardware, evaluate models, open the dashboard
canirunllm scan --no-browser  # same, but skip the dashboard (good for CI/headless)
canirunllm scan --technical   # also print the full technical breakdown in the terminal

canirunllm search qwen        # search the model registry
canirunllm check Qwen3-8B     # check one model or a whole family
canirunllm recommend          # ranked list of models for your hardware

canirunllm web                # launch the dashboard on its own
canirunllm web --port 9000    # on a custom port

What it actually checks

For every model + quantization pair, CanIRunLLM estimates:

• Weight memory from parameter count and quantization (Q4_K_M, Q5_K_M, Q8_0, etc.)
• KV cache size, which grows with context length — a model isn't just "fits" or "doesn't," it fits at a given context length
• Runtime overhead and a safety margin, not just the raw weight size
• Memory strategy: does it fit on a single GPU, does it need to be split across multiple GPUs, or does it need CPU/RAM offloading — each of these is a real, different scenario, not a single generic "offload" verdict
• Runtime compatibility: is the declared runtime (llama.cpp, etc.) actually known to support that memory strategy
• Estimated throughput (tokens/sec), clearly labeled as a modelled estimate, not a benchmark

The result is always a verdict plus a confidence level and a plain-English reason — never a bare "cannot run" with no explanation.

Download & run models directly

Beyond telling you what to run, CanIRunLLM can download a model and run it for you, then let you try it right there in the dashboard's chat box.

Click Download & Run on a model and pick a backend — whichever one you actually have installed shows first and clickable; anything missing shows disabled with a link to get it, instead of failing after the fact:

• Ollama — if Ollama is installed and running, this pulls the model through Ollama itself and serves it from there. Only offered for models with a tag someone has actually checked against Ollama's library (see backends/ollama_tags.py) — a partial, honest list beats a guessed tag that turns out not to exist.
• llama.cpp — downloads the model's GGUF file straight from Hugging Face and serves it with your own llama-server (install llama.cpp and put it on PATH, or point the LLAMA_SERVER_PATH environment variable at the binary). Covers every model in the registry, since the exact source repo for each one is already documented in SOURCES.md.

A couple of things worth knowing going in: gated Hugging Face repos (Llama, Gemma) need your own token in the HF_TOKEN environment variable — there's no way around that automatically, and this tool won't pretend otherwise. And if the app is forcefully killed rather than closed normally, a spawned llama-server process can be left running in the background; see ROADMAP.md for that and everything else that's built, tested, and explicitly left as a known gap rather than glossed over.

Architecture

CLI / Web Dashboard  (presentation only)
        │
Application Layer   (ScannerService, RecommendationEngine)
        │
   ┌────┴─────┬──────────────┬─────────────┐
   ▼          ▼              ▼             ▼
Hardware   Model Registry  Compatibility  Performance
Scanner    + Resolver      Engine         Prediction

The CLI and the local web dashboard are two presentation layers over the same Python core — nothing about compatibility is calculated twice, and the frontend never re-derives a verdict on its own.

The recommendation pipeline is entirely deterministic: hardware detection, memory math, and verdicts are plain Python, not an LLM call. This is a deliberate choice, not a missing feature — a tool whose job is helping you get your first local model can't require one to already be running, and a cloud API would contradict the "no API keys, no accounts" point of the whole project.

Current limitations (being upfront about them)

• GPU detection currently only recognizes NVIDIA GPUs (by calling nvidia-smi directly). On AMD/Intel/Apple Silicon machines it safely falls back to CPU-only mode rather than crashing, but it won't report real GPU numbers yet.
• The model registry is a curated set of well-known open-weight models, not an exhaustive mirror of every model on Hugging Face — see src/canirunllm/registry/SOURCES.md for exactly where every number in it came from.
• Performance numbers are a coarse, clearly-labeled estimate based on parameter count and memory strategy — there is no real benchmarking yet, and the tool never presents an estimate as a measurement.
• Download & Run has been built and tested end-to-end on Windows against real Ollama and llama.cpp installs, but not yet run on macOS or Linux — the underlying code is plain, portable Python, but that's a code-review judgment, not a demonstrated one yet. See ROADMAP.md for the full, current list of what's verified versus what's still open.

Contributing

Contributions are welcome, especially:

• New model registry entries — add an entry to src/canirunllm/registry/models.json and document the exact source for every field (HF config.json URL, GGUF repo, file sizes) in SOURCES.md. Never fabricate a number — if something can't be verified, it doesn't go in.
• AMD/Intel/Apple Silicon GPU detection — the hardware scanner (src/canirunllm/hardware/) currently only supports NVIDIA.
• Bug reports — via GitHub Issues.

Before opening a PR, run the test suite (see below) and make sure it passes.

Testing

pip install -e .
pytest -v

License

MIT — see LICENSE.