llmready 0.1.0

A CLI tool to check if your local hardware can run a Hugging Face LLM.

llmready

llmready is a CLI tool and Python library that analyzes your local system hardware (CPU RAM, Apple Silicon Unified Memory, and NVIDIA GPU VRAM) and accepts a Hugging Face model ID. It calculates the memory required to run that model at various quantizations (FP16, INT8, INT4) and outputs a clear "Go/No-Go" verdict.

It does not download the massive model weights to check memory; it smartly fetches only the metadata via the Hugging Face API!

Features

• Hardware Profiling: Detects Apple Silicon Unified Memory, NVIDIA VRAM, and standard System RAM.
• Fast Metadata Fetching: Retrieves model parameter counts quickly without downloading weights.
• Accurate Memory Math: Includes KV Cache / context window overheads (20%).
• Beautiful Output: Uses rich to print colorful terminal tables with clear Go/No-Go hardware verdicts.

Installation

You can install this package locally using pip:

# Clone the repository and cd into it
cd llmready

# Install the package and its dependencies in editable mode
pip install -e .

This will automatically install dependencies like psutil, huggingface-hub, and rich, and will register the llmready (and legacy llm-check) CLI command.

Usage

Simply run the command with any Hugging Face model repository ID:

llmready meta-llama/Llama-3.2-1B

Advanced Features

1. Context Window Size

By default, the tool calculates the KV Cache memory footprint for an 8192 token context window using the model's exact architecture. You can change this using --context-length or -c:

llmready meta-llama/Llama-2-7b-chat-hf -c 32000

2. Training / Fine-Tuning Memory

If you want to know if you can train a model (Full Fine-Tuning or LoRA/QLoRA), use the --train flag. This calculates the memory required for optimizer states and gradients rather than just inference.

3. Hardware Model Recommendations

Not sure what to run? Ask the tool to scan your hardware and recommend popular models that will fit locally:

llmready --recommend

Private & Gated Models

If you are trying to check a gated model (like meta-llama/Llama-3-8b) or a private repo, you must have access to it on Hugging Face and provide your access token.

You can set the standard Hugging Face token environment variable before running the tool:

export HF_TOKEN="hf_your_access_token_here"
llmready meta-llama/Llama-3-8b

How It Works

1. Hardware: Uses psutil and nvidia-smi (if applicable) to see what compute you have available.
2. Parameters: Uses huggingface_hub to read safetensors.total metadata (or fallback size approximation).
3. Calculation: Memory = Parameters * Bytes per Parameter (FP16=2, INT8=1, INT4=0.6). It also adds a 20% overhead to account for KV caching and memory footprint.
4. Verdict: Compares the required memory to your available VRAM and RAM to issue a 🟢 Pass, 🟡 Warning, or 🔴 Fail.