llamajuice 0.1.0

Squeeze every last drop from your NVIDIA GPU.

llamajuice

Squeeze every last drop from your NVIDIA GPU.

One command catches every local LLM setup issue. Another serves your model with full GPU offload. A third routes requests to the right model automatically.

Born from 16 hours of GPU detection failures, model incompatibilities, and streaming bugs while building an AI agent on an RTX 5050 8GB laptop.

pip install llamajuice

---

juice doctor

The killer feature. Catches every issue that wastes hours.

$ juice doctor

  [ OK  ] GPU: NVIDIA GeForce RTX 5050 Laptop GPU (8.0 GB VRAM)
  [ OK  ] Driver: v572.97
  [ OK  ] CUDA: v12.8
  [ OK  ] VRAM free: 7.4 GB free of 8.0 GB
  [ OK  ] CUDA_VISIBLE_DEVICES: set to '0' (registry)
  [ OK  ] Ollama: running on :11434
  [ OK  ] LM Studio: running on :1234
  [ OK  ]   qwen2.5:7b: 4.4 GB, tool calling supported
  [ WARN]   gemma3:4b: 3.1 GB, no tool calling support
            For agents, use qwen2.5:7b or llama3.1:8b instead.
  [ OK  ] Local GGUFs: 12 models found
  [ OK  ] llama-server: installed, CUDA backend found
  [ OK  ] faster-whisper: installed
  [ OK  ] cuBLAS (Whisper): found via pip (nvidia-cublas-cu12)

  16 passed, 1 warnings, 0 failures

Every warning includes an actionable fix. Use --json for machine-readable output that AI agents can parse and act on.

---

juice serve

Serve any model with full GPU offload. Finds models from LM Studio and Ollama automatically.

$ juice serve qwen2.5:7b

  GPU:    NVIDIA GeForce RTX 5050 Laptop GPU  8.0 GB
  Model:  qwen2.5:7b  (4.4 GB, ollama)
  Loading model to GPU (first load takes a few seconds)...
  VRAM:   4.7/8.0 GB  [#########---]  59%
  Server: http://localhost:11434/v1  (Ollama)

Works with Ollama models, LM Studio GGUFs, or direct paths:

juice serve qwen2.5:7b            # Ollama model
juice serve gemma-3-4b            # fuzzy matches LM Studio GGUF
juice serve ~/models/model.gguf   # direct path, launches llama-server

---

juice combo

Multi-model routing behind one endpoint. Simple questions go to a fast 1.5B model. Complex reasoning and tool calls go to a capable 7B. Your app sees one endpoint.

$ juice combo

  Fast model:  qwen2.5:1.5b
  Main model:  qwen2.5:7b
  Backend:     http://localhost:11434/v1
  Proxy:       http://localhost:8080/v1

  Simple tasks -> qwen2.5:1.5b (instant)
  Complex/tools -> qwen2.5:7b (accurate)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")
# That's it. Your existing code works.
# Juice routes to the right model automatically.

---

juice bench

Benchmark on YOUR hardware. Compare servers side by side.

$ juice bench

  Hardware: NVIDIA GeForce RTX 5050 Laptop GPU (8.0 GB)

  +---------------------------------------------------------------+
  | Server | Model      | tok/s |  TTFT |             VRAM | Tools |
  |--------+------------+-------+-------+------------------+-------|
  | Ollama | qwen2.5:7b |  13.2 | 652ms | 4.7/8.0 GB (59%) | pass |
  +---------------------------------------------------------------+

  Ollama: 13 tok/s / 652ms TTFT / 59% GPU / tools:pass on RTX 5050

Use --json for machine-readable output.

---

All commands

Command	What it does
juice doctor	Full diagnostic — GPU, drivers, models, dependencies. --json for agents.
juice serve <model>	Launch with full GPU offload. Ollama, LM Studio, or GGUF path.
juice combo	Multi-model proxy — fast + main model, one endpoint.
juice bench	Benchmark tok/s, TTFT, VRAM, tool calls. --json for agents.
juice status	GPU info + detected servers.
juice models	Your local models (LM Studio + Ollama).
juice recommend	Suggested models for your VRAM tier.
juice setup	Download llama-server binary.

---

Why not just use Ollama?

Ollama is great. Use it. Juice sits alongside it and fixes the gaps:

• GPU detection fails silently on RTX 50-series. juice doctor catches this.
• No way to verify your model is actually on GPU. juice doctor checks VRAM.
• No multi-model routing. juice combo gives you a fast model + a smart model behind one endpoint.
• Models without tool support silently fail. juice doctor tells you which models support tool calling.

Juice doesn't replace Ollama. It makes it reliable.

---

For AI agents

juice doctor --json outputs structured diagnostics that agents can parse and fix:

{
  "check": "CUDA_VISIBLE_DEVICES",
  "status": "WARN",
  "message": "not set (RTX 50-series detected)",
  "fixes": [
    {
      "command": "setx CUDA_VISIBLE_DEVICES 0",
      "platform": "win32",
      "requires_restart": true,
      "destructive": false
    }
  ]
}

Add to your project's CLAUDE.md:

Run `juice doctor --json` and fix any warnings where destructive=false.

---

Install

pip install llamajuice

Requires:

• Python 3.10+
• NVIDIA GPU
• Ollama or LM Studio (for models)

On first juice setup, the correct llama-server binary for your CUDA version is downloaded automatically (~220 MB, one time).

---

Licence

MIT