llama-index-llms-mlx 0.3.0

llama-index llms mlx integration

LlamaIndex Llms Integration: MLX

Overview

---

Integrate with MLX LLMs from the mlx-lm library

Installation

---

pip install llama-index-llms-mlx

Example

---

from llama_index.llms.mlx import MLXLLM

llm = MLXLLM(
    model_name="microsoft/phi-2",
    context_window=3900,
    max_new_tokens=256,
    generate_kwargs={"temp": 0.7, "top_p": 0.95},
)

response = llm.complete("What is the meaning of life?")
print(str(response))