ai-launcher-cli 0.1.6

Simple CLI to chat with GGUF models locally (no Ollama/LM Studio required)

ai-launcher-cli

Simple Python CLI to chat with .gguf models locally using llama-cpp-python. No Ollama, llama.cpp, LM Studio, or online providers required.

Install

pip install ai-launcher-cli

Usage

# Basic usage
ailaunch path/to/model.gguf

# With custom settings
ailaunch model.gguf -c 8192 -t 0.8 --max-tokens 1024

# Disable streaming (wait for full response)
ailaunch model.gguf --no-stream

# Custom system prompt
ailaunch model.gguf --system "You are a coding assistant."

# Use a built-in system prompt template
ailaunch model.gguf --system-template coder

# List available models
ailaunch --list-models

# Auto-select model from common directories
ailaunch auto

# Options:
#   -c, --ctx-size      Context window size (default: 4096)
#   -g, --gpu-layers    GPU layers to offload (-1 = all, default: -1)
#   -t, --threads       CPU threads (0 = auto, default: 0)
#   --temperature        Sampling temperature (default: 0.7)
#   --max-tokens        Max tokens to generate (default: 512)
#   --no-stream         Disable streaming output
#   --system            Custom system prompt
#   --system-template   Built-in template (coder, reviewer, teacher, creative, analyst, translator, shell)
#   --list-models       List available GGUF models and exit
#   --save-config       Save current options as defaults
#   --benchmark         Run benchmark after loading
#   --export            Export conversation on exit (markdown/json)
#   --export-file       File to export conversation to
#   --no-history        Disable loading/saving chat history
#   --clear-history     Clear chat history for this model
#   -v, --version       Show version

Interactive Commands

While chatting, type any of these commands:

Command	Description
/help	Show help
/save	Save conversation to history
/export [fmt]	Export conversation (markdown/json)
/clear	Clear conversation (keep system prompt)
/system <prompt>	Change system prompt
/template <name>	Use built-in template
/config	Show current configuration
/bench	Run benchmark
/models	List available models
/switch [path]	Switch to another model
exit/quit/q	Exit

Configuration

Config is saved to ~/.config/ailaunch/config.yaml. Use --save-config to save current options.

Model Auto-Detection

Models are automatically searched in these directories:

• ~/.lmstudio/models
• ~/.lmstudio/.internal/bundled-models
• ~/.cache/huggingface/hub
• ~/models
• ~/Downloads
• ~/OneDrive/Downloads
• ~/OneDrive/Documents/Downloads

GPU Acceleration

Install with GPU extras for acceleration:

# NVIDIA CUDA
pip install ai-launcher-cli[cuda]

# Apple Metal
pip install ai-launcher-cli[metal]

Then use -g -1 to offload all layers to GPU.

Requirements

• Python 3.8+
• llama-cpp-python>=0.3.0 (installs automatically)

Exit

Type exit, quit, q or press Ctrl+C to exit.