llmpm 1.1.0

LLM Package Manager — download, run, and share AI models from the command line

llmpm — LLM Package Manager

Command-line package manager for open-source large language models. Download, run, and share LLMs with a single command.

llmpm is a package manager for Large Language models, inspired by pip & npm. Models are sourced from HuggingFace Hub. Supports text generation (GGUF via llama.cpp and Transformer checkpoints), image generation (Diffusion), vision, speech-to-text (ASR), and text-to-speech (TTS).

---

Installation

via pip (recommended)

pip install llmpm

via npm

npm install -g llmpm

The npm package automatically installs the Python backend via pip.

---

Quick start

# Install a GGUF model (interactive quantisation picker)
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF

# Run it
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF

---

Commands

Command	Description
llmpm install <repo>	Download and install a model from HuggingFace
llmpm run <repo>	Run an installed model (interactive chat)
llmpm serve <repo>	Serve a model as an OpenAI-compatible HTTP API
llmpm push <repo>	Upload a model to HuggingFace Hub
llmpm list	Show all installed models
llmpm info <repo>	Show details about a model
llmpm uninstall <repo>	Uninstall a model

---

llmpm install

# Interactively choose a GGUF quantisation
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF

# Install a specific quantisation
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --quant Q4_K_M

# Install a full Transformer model
llmpm install meta-llama/Llama-3.2-1B-Instruct

# Install a single specific file
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --file Llama-3.2-1B-Instruct-Q4_K_M.gguf

# Skip prompts (pick best default)
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --no-interactive

Models are stored in ~/.llmpm/models/.

---

llmpm run

llmpm run auto-detects the model type and launches the appropriate interactive session. It supports text generation, image generation, vision, speech-to-text (ASR), and text-to-speech (TTS) models.

Text generation (GGUF & Transformers)

# Interactive chat
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF

# Single-turn inference
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --prompt "Explain quantum computing"

# With a system prompt
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --system "You are a helpful pirate."

# Limit response length
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --max-tokens 512

# GGUF: tune context window and GPU layers
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --ctx 8192 --gpu-layers 32

# Transformer model
llmpm run HuggingFaceTB/SmolLM2-1.7B-Instruct

Image generation (Diffusion)

Generates an image from a text prompt and saves it as a PNG on your Desktop.

# Single prompt → saves llmpm_<timestamp>.png to ~/Desktop
llmpm run amused/amused-256 --prompt "a cyberpunk city at sunset"

# Interactive session (type a prompt, get an image each time)
llmpm run amused/amused-256

In interactive mode type your prompt and press Enter. The output path is printed after each generation. Type /exit to quit.

Requires: pip install diffusers torch accelerate

Vision (image-to-text)

Describe or answer questions about an image. Pass the image file path via --prompt.

# Single image description
llmpm run Salesforce/blip-image-captioning-base --prompt /path/to/photo.jpg

# Interactive session: type an image path at each prompt
llmpm run Salesforce/blip-image-captioning-base

Requires: pip install transformers torch Pillow

Speech-to-text / ASR

Transcribe an audio file. Pass the audio file path via --prompt.

# Transcribe a single file
llmpm run openai/whisper-base --prompt recording.wav

# Interactive: enter an audio file path at each prompt
llmpm run openai/whisper-base

Supported formats depend on your installed audio libraries (wav, flac, mp3, …).

Requires: pip install transformers torch

Text-to-speech / TTS

Convert text to speech. The output WAV file is saved to your Desktop.

# Single utterance → saves llmpm_<timestamp>.wav to ~/Desktop
llmpm run suno/bark-small --prompt "Hello, how are you today?"

# Interactive session
llmpm run suno/bark-small

Requires: pip install transformers torch

llmpm run options

Option	Default	Description
--prompt / -p	—	Single-turn prompt or input file path (non-interactive)
--system / -s	—	System prompt (text generation only)
--max-tokens	128000	Maximum tokens to generate per response
--ctx	128000	Context window size (GGUF only)
--gpu-layers	-1	GPU layers to offload, -1 = all (GGUF only)
--verbose	off	Show model loading output

Interactive session commands

These commands work in any interactive session:

Command	Action
/exit	End the session
/clear	Clear conversation history (text gen only)
/system <text>	Update the system prompt (text gen only)

Model type detection

llmpm run reads config.json / model_index.json from the installed model to determine the pipeline type before loading any weights. The detected type is printed at startup:

  Detected: Image Generation (Diffusion)
  Loading model…  ✓

If detection is ambiguous the model falls back to the text-generation backend.

---

llmpm serve

Start a local HTTP server exposing the model as an OpenAI-compatible REST API. A browser-based chat UI is also available at /chat.

# Serve on the default port (8080)
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF

# Custom port and host
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --port 9000 --host 0.0.0.0

# Set the default max tokens (clients may override per-request)
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --max-tokens 2048

# GGUF options
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --ctx 8192 --gpu-layers 32

llmpm serve options

Option	Default	Description
--port / -p	8080	Port to listen on (auto-increments if busy)
--host / -H	localhost	Host/address to bind to
--max-tokens	128000	Default max tokens per response (overridable per-request)
--ctx	128000	Context window size (GGUF only)
--gpu-layers	-1	GPU layers to offload, -1 = all (GGUF only)

Endpoints

Method	Path	Models	Description
GET	/chat	all	Browser chat / image-gen UI
GET	/health	all	{"status":"ok","model":"<id>"}
POST	/v1/chat/completions	text-gen	OpenAI-compatible chat inference (SSE streaming supported)
POST	/v1/images/generations	diffusion	Text-to-image; pass image (base64) for image-to-image
POST	/v1/audio/transcriptions	ASR	Speech-to-text
POST	/v1/audio/speech	TTS	Text-to-speech

Example API calls

# Text generation
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 256}'

# Text-to-image
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a cat in a forest", "n": 1}'

# Image-to-image (include the source image as base64 in the same endpoint)
IMAGE_B64=$(base64 -i input.png)
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d "{\"prompt\": \"turn it into a painting\", \"image\": \"$IMAGE_B64\"}"

Response shape (both text-to-image and image-to-image):

{
  "created": 1234567890,
  "data": [{ "b64_json": "<base64-png>" }]
}

---

llmpm push

# Push an already-installed model
llmpm push my-org/my-fine-tune

# Push a local directory
llmpm push my-org/my-fine-tune --path ./my-model-dir

# Push as private repository
llmpm push my-org/my-fine-tune --private

# Custom commit message
llmpm push my-org/my-fine-tune -m "Add Q4_K_M quantisation"

Requires a HuggingFace token (run huggingface-cli login or set HF_TOKEN).

---

Backends

Model type	Pipeline	Backend	Extra install
.gguf files	Text generation	llama.cpp via llama-cpp-python	pip install llmpm[gguf]
.safetensors / .bin	Text generation	HuggingFace Transformers	pip install llmpm[transformers]
Diffusion models	Image generation	HuggingFace Diffusers	pip install llmpm[diffusion]
Vision models	Image-to-text	HuggingFace Transformers	pip install llmpm[vision]
Whisper / ASR models	Speech-to-text	HuggingFace Transformers	pip install llmpm[audio]
TTS models	Text-to-speech	HuggingFace Transformers	pip install llmpm[audio]

Install all backends at once:

pip install llmpm[all]

---

Configuration

Variable	Default	Description
LLMPM_HOME	~/.llmpm	Root directory for models and registry
HF_TOKEN	—	HuggingFace API token for gated models
LLPM_PYTHON	python3	Python binary used by the npm shim

---

License

MIT