llmpm 1.2.0

LLM Package Manager — download, run, and share AI models from the command line

llmpm — LLM Package Manager

Command-line package manager for open-source large language models. Download, run, and share LLMs with a single command.

llmpm is a package manager for Large Language models, inspired by pip & npm. Models are sourced from HuggingFace Hub. Supports text generation (GGUF via llama.cpp and Transformer checkpoints), image generation (Diffusion), vision, speech-to-text (ASR), and text-to-speech (TTS).

---

Installation

via pip (recommended)

pip install llmpm

via npm

npm install -g llmpm

The npm package automatically installs the Python backend via pip.

---

Quick start

# Install a model
llmpm install meta-llama/Llama-3.2-3B-Instruct

# Run it
llmpm run meta-llama/Llama-3.2-3B-Instruct

---

Commands

Command	Description
llmpm install <repo>	Download and install a model from HuggingFace
llmpm run <repo>	Run an installed model (interactive chat)
llmpm serve [repo] [repo] ...	Serve one or more models as an OpenAI-compatible API
llmpm serve	Serve every installed model on a single HTTP server
llmpm push <repo>	Upload a model to HuggingFace Hub
llmpm list	Show all installed models
llmpm info <repo>	Show details about a model
llmpm uninstall <repo>	Uninstall a model

---

llmpm install

# Install a Transformer model
llmpm install meta-llama/Llama-3.2-3B-Instruct

# Install a GGUF model (interactive quantisation picker)
llmpm install bartowski/Llama-3.2-3B-Instruct-GGUF

# Install a specific GGUF quantisation
llmpm install bartowski/Llama-3.2-3B-Instruct-GGUF --quant Q4_K_M

# Install a single specific file
llmpm install bartowski/Llama-3.2-3B-Instruct-GGUF --file Llama-3.2-3B-Instruct-Q4_K_M.gguf

# Skip prompts (pick best default)
llmpm install meta-llama/Llama-3.2-3B-Instruct --no-interactive

Models are stored in ~/.llmpm/models/.

---

llmpm run

llmpm run auto-detects the model type and launches the appropriate interactive session. It supports text generation, image generation, vision, speech-to-text (ASR), and text-to-speech (TTS) models.

Text generation (GGUF & Transformers)

# Interactive chat
llmpm run meta-llama/Llama-3.2-3B-Instruct

# Single-turn inference
llmpm run meta-llama/Llama-3.2-3B-Instruct --prompt "Explain quantum computing"

# With a system prompt
llmpm run meta-llama/Llama-3.2-3B-Instruct --system "You are a helpful pirate."

# Limit response length
llmpm run meta-llama/Llama-3.2-3B-Instruct --max-tokens 512

# GGUF model — tune context window and GPU layers
llmpm run bartowski/Llama-3.2-3B-Instruct-GGUF --ctx 8192 --gpu-layers 32

Image generation (Diffusion)

Generates an image from a text prompt and saves it as a PNG on your Desktop.

# Single prompt → saves llmpm_<timestamp>.png to ~/Desktop
llmpm run amused/amused-256 --prompt "a cyberpunk city at sunset"

# Interactive session (type a prompt, get an image each time)
llmpm run amused/amused-256

In interactive mode type your prompt and press Enter. The output path is printed after each generation. Type /exit to quit.

Requires: pip install diffusers torch accelerate

Vision (image-to-text)

Describe or answer questions about an image. Pass the image file path via --prompt.

# Single image description
llmpm run Salesforce/blip-image-captioning-base --prompt /path/to/photo.jpg

# Interactive session: type an image path at each prompt
llmpm run Salesforce/blip-image-captioning-base

Requires: pip install transformers torch Pillow

Speech-to-text / ASR

Transcribe an audio file. Pass the audio file path via --prompt.

# Transcribe a single file
llmpm run openai/whisper-base --prompt recording.wav

# Interactive: enter an audio file path at each prompt
llmpm run openai/whisper-base

Supported formats depend on your installed audio libraries (wav, flac, mp3, …).

Requires: pip install transformers torch

Text-to-speech / TTS

Convert text to speech. The output WAV file is saved to your Desktop.

# Single utterance → saves llmpm_<timestamp>.wav to ~/Desktop
llmpm run suno/bark-small --prompt "Hello, how are you today?"

# Interactive session
llmpm run suno/bark-small

Requires: pip install transformers torch

llmpm run options

Option	Default	Description
--prompt / -p	—	Single-turn prompt or input file path (non-interactive)
--system / -s	—	System prompt (text generation only)
--max-tokens	128000	Maximum tokens to generate per response
--ctx	128000	Context window size (GGUF only)
--gpu-layers	-1	GPU layers to offload, -1 = all (GGUF only)
--verbose	off	Show model loading output

Interactive session commands

These commands work in any interactive session:

Command	Action
/exit	End the session
/clear	Clear conversation history (text gen only)
/system <text>	Update the system prompt (text gen only)

Model type detection

llmpm run reads config.json / model_index.json from the installed model to determine the pipeline type before loading any weights. The detected type is printed at startup:

  Detected: Image Generation (Diffusion)
  Loading model…  ✓

If detection is ambiguous the model falls back to the text-generation backend.

---

llmpm serve

Start a single local HTTP server exposing one or more models as an OpenAI-compatible REST API. A browser-based chat UI is available at /chat.

# Serve a single model on the default port (8080)
llmpm serve meta-llama/Llama-3.2-3B-Instruct

# Serve multiple models on one server
llmpm serve meta-llama/Llama-3.2-3B-Instruct amused/amused-256

# Serve ALL installed models automatically
llmpm serve

# Custom port and host
llmpm serve meta-llama/Llama-3.2-3B-Instruct --port 9000 --host 0.0.0.0

# Set the default max tokens (clients may override per-request)
llmpm serve meta-llama/Llama-3.2-3B-Instruct --max-tokens 2048

# GGUF model — tune context window and GPU layers
llmpm serve bartowski/Llama-3.2-3B-Instruct-GGUF --ctx 8192 --gpu-layers 32

Fuzzy model-name matching is applied to each argument — if multiple installed models match you will be prompted to pick one.

llmpm serve options

Option	Default	Description
--port / -p	8080	Port to listen on (auto-increments if busy)
--host / -H	localhost	Host/address to bind to
--max-tokens	128000	Default max tokens per response (overridable per-request)
--ctx	128000	Context window size (GGUF only)
--gpu-layers	-1	GPU layers to offload, -1 = all (GGUF only)

Multi-model routing

When multiple models are loaded, POST endpoints accept an optional "model" field in the JSON body. If omitted, the first loaded model is used.

# Target a specific model when multiple are loaded
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Llama-3.2-3B-Instruct",
       "messages": [{"role": "user", "content": "Hello!"}]}'

The chat UI at /chat shows a model dropdown when more than one model is loaded. Switching models resets the conversation and adapts the UI to the new model's category.

Endpoints

Method	Path	Description
GET	/chat	Browser chat / image-gen UI (model dropdown for multi-model serving)
GET	/health	{"status":"ok","models":["id1","id2",…]}
GET	/v1/models	List all loaded models with id, category, created
GET	/v1/models/<id>	Info for a specific loaded model
POST	/v1/chat/completions	OpenAI-compatible chat inference (SSE streaming supported)
POST	/v1/completions	Legacy text completion
POST	/v1/embeddings	Text embeddings
POST	/v1/images/generations	Text-to-image; pass "image" (base64) for image-to-image
POST	/v1/audio/transcriptions	Speech-to-text
POST	/v1/audio/speech	Text-to-speech

All POST endpoints accept "model": "<id>" to target a specific loaded model.

Example API calls

# Text generation (streaming)
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}],
       "max_tokens": 256, "stream": true}'

# Target a specific model when multiple are loaded
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Llama-3.2-1B-Instruct",
       "messages": [{"role": "user", "content": "Hello!"}]}'

# List all loaded models
curl http://localhost:8080/v1/models

# Text-to-image
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a cat in a forest", "n": 1}'

# Image-to-image (include the source image as base64 in the same endpoint)
IMAGE_B64=$(base64 -i input.png)
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d "{\"prompt\": \"turn it into a painting\", \"image\": \"$IMAGE_B64\"}"

# Speech-to-text
curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -H "Content-Type: application/octet-stream" \
  --data-binary @recording.wav

# Text-to-speech
curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello world"}' \
  --output speech.wav

Response shape for image generation:

{
  "created": 1234567890,
  "data": [{ "b64_json": "<base64-png>" }]
}

---

llmpm push

# Push an already-installed model
llmpm push my-org/my-fine-tune

# Push a local directory
llmpm push my-org/my-fine-tune --path ./my-model-dir

# Push as private repository
llmpm push my-org/my-fine-tune --private

# Custom commit message
llmpm push my-org/my-fine-tune -m "Add Q4_K_M quantisation"

Requires a HuggingFace token (run huggingface-cli login or set HF_TOKEN).

---

Backends

Model type	Pipeline	Backend	Extra install
.gguf files	Text generation	llama.cpp via llama-cpp-python	pip install llmpm[gguf]
.safetensors / .bin	Text generation	HuggingFace Transformers	pip install llmpm[transformers]
Diffusion models	Image generation	HuggingFace Diffusers	pip install llmpm[diffusion]
Vision models	Image-to-text	HuggingFace Transformers	pip install llmpm[vision]
Whisper / ASR models	Speech-to-text	HuggingFace Transformers	pip install llmpm[audio]
TTS models	Text-to-speech	HuggingFace Transformers	pip install llmpm[audio]

Install all backends at once:

pip install llmpm[all]

---

Configuration

Variable	Default	Description
LLMPM_HOME	~/.llmpm	Root directory for models and registry
HF_TOKEN	—	HuggingFace API token for gated models
LLPM_PYTHON	python3	Python binary used by the npm shim

---

License

MIT