tinillm 2.3.0

Hardware LLM capability scanner — know what runs on your machine

tinillm

Know what LLMs your hardware can run — locally, instantly.

pipx install tinillm
tinillm

---

What it does

tinillm is an interactive tool. Launch it by typing tinillm in your terminal, and you'll land in a welcome screen. From there, every feature is a slash command: /scan inspects your hardware, /models browses real LLMs, /run launches one in Ollama, and so on.

╭─── tinillm v1.9.0 ───────────────────────────────────────────────────╮
│  ████████╗                                                           │
│     ██╔══╝    Welcome back, Harish!      Tips for getting started    │
│     ██║                                  ────────────────────────    │
│     ██║       v1.9.0 · tinillm             Run /scan    to detect    │
│     ╚═╝       ~/tinillm_CLI                Run /models  to browse    │
│               ollama ● running             Run /run     to launch    │
│                                            Run /doctor  health chk   │
│                                            Run /help    list all     │
│                                                                      │
│                                          Recent activity             │
│                                          ────────────────────────    │
│                                            /scan                     │
│                                            /run llama3.2:3b          │
╰──────────────────────────────────────────────────────────────────────╯
  Type /help for commands · /exit or Ctrl+D to quit

tinillm> /scan

  LLM Capability Matrix

  Model    Fit        Best Quant   Mem Needed   Tokens/sec
  ~1B      Perfect    Q8_0          1.9 GB        580 t/s
  ~3B      Perfect    Q8_0          3.8 GB        195 t/s
  ~7B      Perfect    Q6_K          6.2 GB         88 t/s
  ~13B     Perfect    Q5_K_M       10.1 GB         47 t/s
  ~34B     Good       Q4_K_M       21.8 GB         18 t/s

tinillm>

Works on Linux, macOS, and Windows. No GPU required — CPU-only machines are supported too.

---

Install

pipx install tinillm     # recommended: isolated per-tool environment
# or
pip install tinillm

Requires Python 3.11+. No other tools needed.

---

Usage

Launch the tool with a single command:

tinillm

Inside the REPL, every feature is a slash command:

Command	What it does
/scan	Scan hardware and show which LLM sizes fit
/scan --verbose	Include model sizes that don't fit
/scan --json	Machine-readable JSON output
/models	Browse real LLM models and see which fit
/models --fits-only	Hide models that don't fit
/models --ollama	Show which models are installed in local Ollama
/run	Pick a compatible model interactively and run it
/run llama3.2:3b	Launch a specific model directly
/suggest --use-case coding	Personalised model recommendation
/doctor	System health check (hardware + Ollama)
/help	List every command
/clear	Clear the terminal
/exit	Quit (or Ctrl+D)

Tab-completion works on slash commands, subcommands, and flags.

---

First launch

The first time you run tinillm, it automatically runs /scan for you so you see your hardware capabilities immediately. On subsequent launches, just the welcome panel appears.

---

GPU support

Vendor	Detection method
NVIDIA	nvidia-smi → sysfs fallback
AMD	rocm-smi → sysfs fallback
Apple Silicon	system_profiler (unified memory)
Intel Arc	sysfs + lspci
Windows (all)	PowerShell WMI
Any	vulkaninfo last-resort fallback

---

Fit levels explained

Level	Meaning
Perfect	Fits comfortably at Q4_K_M or better with ≥20% VRAM headroom
Good	Fits but tightly
Marginal	Fits only at heavy compression / reduced context, or CPU-only
TooTight	Won't fit under any quantisation

---

Versioning

Version	Feature
1.9	Dropped RAG — focused on hardware scanning + model runner ← current
1.8	Interactive REPL (single entry-point, slash commands)
1.7	Added RAG (/index, /ask, /rag)
1.1	First feature — hardware scanner

---

Part of the tini* family

Tool	What it does
tiniRAG	Privacy-first RAG CLI
tinillm	Interactive LLM + hardware tool