smartpull 0.1.0

Hardware-aware LLM orchestration — zero-config local model setup for any GPU

smartpull ⚡

Hardware-aware LLM orchestration. Zero config. Zero VRAM crashes. Just works.

---

The Problem

Every local LLM runner (Ollama, LM Studio) is hardware-blind.

You have to manually figure out:

• Which quantization fits your VRAM?
• What context window is safe?
• Why is your model swapping to RAM and running at 2 tok/s?

This wastes 30+ minutes every time a new model drops.

---

The Solution

pip install smartpull
smartpull build

🔍 Scanning hardware...
  GPU     : NVIDIA GeForce RTX 3050 Ti Laptop GPU
  Usable  : 2.38 GB (after 15% safety buffer)

✅ Model          : gemma4:e2b
✅ Quantization   : IQ4_XS
✅ Context window : 4,608 tokens
✅ Headroom       : 780 MB
✅ Swap risk      : LOW

📝 Modelfile written to: ./Modelfile

Run these commands:
  ollama pull gemma4:e2b
  ollama create smartpull-gemma4-e2b -f ./Modelfile
  ollama run smartpull-gemma4-e2b

30 minutes → 10 seconds.

---

Install

pip install smartpull

Or run from source:

git clone https://github.com/punvesh/smartpull
cd smartpull
pip install -e ".[dev]"

---

Commands

Command	What it does
smartpull scan	Show your GPU hardware profile
smartpull recommend	Best model for your current VRAM
smartpull build	Generate Modelfile + print ollama commands
smartpull matrix	Show full model matrix reference table

---

How It Works

nvidia-smi
    ↓
Real-time VRAM (free, not total)
    ↓
15% safety buffer applied
    ↓
Model matrix lookup (VRAM → model + quant)
    ↓
MoE awareness check
    ↓
Dynamic context window scaling
    ↓
Jinja2 Modelfile generation
    ↓
ollama create → run

---

Supported Models

VRAM	Model	Quant	CTX
< 1.8 GB	gemma2:2b	IQ4_XS	2,048
1.8–2.8 GB	gemma4:e2b	IQ4_XS	4,096
2.8–4.5 GB	qwen2.5-coder:3b	Q4_K_S	8,192
4.5–6 GB	qwen2.5-coder:7b	Q4_K_S	16,384
6–9 GB	llama3.1:8b	Q5_K_M	32,768
9 GB+	qwen2.5-coder:14b	Q5_K_M	32,768

---

Docker

docker build -t smartpull .
docker run --gpus all smartpull recommend

---

Development

pip install -e ".[dev]"
pytest tests/ -v
black .

---

Roadmap

• NVIDIA GPU support
• Ollama Modelfile generation
• MoE model awareness
• Dynamic context window scaling
• Apple Silicon (MLX) support
• AMD ROCm support
• Claude Code bridge (auto-set ANTHROPIC_BASE_URL)
• GUI dashboard

---

Contributing

PRs welcome. Open an issue first for large changes.

---

License

MIT © Punvesh