bestllm 0.0.1

Tries to pick the best local llm for your computer and environment.

bestllm

Picks the most capable local LLM your machine can comfortably run, with zero guesswork.

bestllm is a hardware-aware companion inspired by Timothy Crosley's developer tools. It inspects your system, compares it against curated model profiles, and returns the right local model for the job—no spreadsheet required.

Highlights

• Understands your hardware with lightweight, cross-platform detection.
• Ships with curated GPU and CPU friendly GGUF profiles and their resource budgets.
• Provides a friendly CLI plus a composable Python API for deeper integrations.
• Fails fast with actionable messages when your machine does not meet a model's needs.
• Designed to be extended: drop in new profiles or swap out detectors without rewriting core logic.

Quick Start

Installation

python -m pip install -e .

Need dev tooling too? Grab the extras:

uv pip install --group dev .

Run the CLI

bestllm
# or
python -m bestllm

Expect output similar to:

Recommended model: llama3-8b-instruct-q4 (8,192 token context window).
Summary: requires >= 16GB RAM, >= 4 CPU cores, 10 GPU VRAM requirement.
Detected GPU VRAM: 12GB — using GPU-friendly profile.

Use from Python

from bestllm import best_local_llm

profile = best_local_llm()
print(profile.name)
print(profile.as_dict())

How It Works

• bestllm.hardware.HardwareSpecs captures total RAM, CPU cores, and GPU VRAM.
• bestllm.models defines the catalog of ModelProfile objects (size, context window, and resource requirements).
• bestllm.selector.recommend_model_for_specs scores the catalog against your hardware and raises a clear error if nothing fits.
• bestllm.__init__ exposes a minimal API and wires everything into the console script entry point.

Extending the Model Catalog

1. Open src/bestllm/models.py and add a ModelProfile to DEFAULT_MODEL_PROFILES.
2. Document resource requirements and noteworthy constraints in the notes field.
3. Re-run bestllm to confirm the new profile appears when the hardware qualifies.
4. Add targeted tests under tests/models/ (mirroring the src/bestllm layout) to lock down the new behavior.

Configuration Tips

• Pair bestllm with your runner scripts to auto-pick the right gguf file before launch.
• Override hardware detection in constrained environments by instantiating HardwareSpecs manually and passing it to recommend_model_for_specs.
• Guard long-context models behind feature flags so CI and lightweight machines stick to CPU options.

Roadmap

• Smarter scoring that accounts for quantization preferences and batching strategies.
• Built-in adapters for popular launchers (Ollama, llama.cpp, vLLM) via optional extras.
• Configurable profile sources (TOML/JSON) so catalogs can live outside the codebase.
• Hardware detection helpers for AMD and Apple GPUs without shelling out to vendor tools.

Contributing

• Ensure ruff check src and ty src pass before sending a PR.
• Keep new features isolated in their own modules under src/bestllm/ and surface them through bestllm.__init__.
• Add or update tests/ alongside feature changes; we mirror the runtime package layout for discovery.
• Share CLI transcripts or screenshots when behavior changes—context speeds up reviews.

Inspiration & Credits

bestllm follows the pragmatic tooling ethos of Timothy Crosley projects: approachable defaults, thoughtful ergonomics, and extensibility baked in. Thanks for exploring, and feel free to shape the future roadmap with issues or discussions.