Vision LLMs on Apple silicon with MLX and the Hugging Face Hub
Project description
MLX-VLM
MLX-VLM a package for running Vision LLMs on your Mac using MLX.
Get started
The easiest way to get started is to install the mlx-vlm
package:
With pip
:
pip install mlx-vlm
Inference
CLI
python -m mlx_vlm.generate --model qnguyen3/nanoLLaVA --max-tokens 100 --temp 0.0
Chat UI with Gradio
python -m mlx_vlm.chat_ui --model qnguyen3/nanoLLaVA
Script
import mlx.core as mx
from mlx_vlm import load, generate
model_path = "mlx-community/llava-1.5-7b-4bit"
model, processor = load(model_path)
prompt = processor.apply_chat_template(
[{"role": "user", "content": f"<image>\nWhat are these?"}],
tokenize=False,
add_generation_prompt=True,
)
output = generate(model, processor, "http://images.cocodataset.org/val2017/000000039769.jpg", prompt, verbose=False)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
mlx_vlm-0.0.5.tar.gz
(28.8 kB
view hashes)
Built Distribution
mlx_vlm-0.0.5-py3-none-any.whl
(39.8 kB
view hashes)