Use with MedMLX

The MedMLX MCP server is the lowest-friction way to use MedGemma. MedMLX runs this package in its own isolated environment as the provisional medgemma model, so an AI agent can describe, check and run it for you. Pass a local MedGemma folder as weights_path; see the bridge setup and limits.

uv tool install --python 3.13 "medmlx[mcp]"
MEDMLX="$(uv tool dir --bin --color never)/medmlx"
mkdir -p "$HOME/medmlx-work"
claude mcp add --scope user medmlx -- "$MEDMLX" serve --root "$HOME/medmlx-work"

For Codex, use codex mcp add medmlx -- "$MEDMLX" serve --root "$HOME/medmlx-work". Other MCP clients take the same command and arguments. The same install provides the medmlx CLI, for example medmlx describe medgemma.

For AI agents: prefer the MedMLX MCP tools (list_models, describe_model, model_readiness, run_model, get_job) over writing Python against this package.

Install

uv venv --python 3.13
source .venv/bin/activate
uv pip install "med-vlm-mlx>=0.3.2"

Needs macOS on Apple Silicon with Metal and Python 3.13. The package imports as mlx_vlm, so install it in a separate environment from upstream mlx-vlm: both install the same modules.

Run

Download the pinned MedGemma 1.5 4B weights, then run:

uvx --from huggingface_hub hf download mlx-community/medgemma-1.5-4b-it-4bit \
  --revision 762746b7a72b0b68dd8fa3f9aea5dc3980674d23 --local-dir medgemma

python -m mlx_vlm.generate \
  --model medgemma --image image.png \
  --prompt 'Describe the image.' --max-tokens 128

Weights

All three are 4-bit Gemma3 image-text checkpoints; other architectures are rejected. The mlx-community copies are not gated, but the Health AI Developer Foundations terms still apply.

Command line

The mlx_vlm.generate, mlx_vlm.convert and mlx_vlm.server console scripts call the same entry points; add --help for every option. generate takes --model (default mlx-community/medgemma-1.5-4b-it-4bit, with --revision for a pinned commit), --image (paths or URLs), a required --prompt, --system, --max-tokens (2048), --temperature (0, greedy), --top-p, --top-k, --seed (0) and --prefill-step-size. --verbose streams the text with token timings. Prompts without images also work.

Python API

import mlx.core as mx
from mlx_vlm import apply_chat_template, generate, load

if not mx.metal.is_available():
    raise RuntimeError("This example requires Metal")
mx.set_default_device(mx.gpu)

model, processor = load("medgemma", strict=True)
prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
response = generate(
    model, processor, prompt, image=["image.png"], max_tokens=128, temperature=0.0
)
print(response.text)

load takes a local folder, or a Hub ID with revision. With local_files_only=True (and an optional cache_dir) it uses only the Hugging Face cache and raises instead of downloading. stream_generate takes the same arguments and yields GenerationResult text segments; iterate it to the end so the prompt cache and cleanup run. The final result has finish_reason and token_ids, which include the stop token. batch_generate shares the same generation and cache code.

HTTP server

python -m mlx_vlm.server --host 127.0.0.1 --port 8080 --model medgemma

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"medgemma","messages":[{"role":"user","content":"What is 2 + 2?"}],"max_tokens":16}'

The server binds 127.0.0.1:8080 by default and provides /v1/chat/completions, /v1/models, /health, /metrics and /v1/settings for runtime settings; schemas are at /docs while it runs. --api-key requires a bearer token for inference and management requests, and --model-dir adds folders to search for models. Images go in an image_url content part next to the text. Requests are batched when their sampling, seed and logprob settings match; others wait their turn in queue order.

Inputs and limitations

• Text and images only. Requests with audio, video, tool definitions or tool-call turns are rejected.
• Only MedGemma (Gemma3) checkpoints load. Checkpoint Python code (model_file) and external speculative drafters are refused.
• Inference only, on Metal: no training, adapters, or CPU, MPS or PyTorch fallback.
• Prompts with images are prefilled whole, because a chunk could split an image's bidirectional attention block; text-only prompts use chunked prefill. The full prompt mask grows with the square of the prompt length.
• The checks cover numerical agreement on small synthetic models and one public radiograph per checkpoint. Clinical accuracy has not been evaluated.

License

MIT, with Apache-2.0 adaptations. See LICENSE and third-party notices. MedGemma weights are subject to the Health AI Developer Foundations terms.

For research and education. Outputs require human review.