llama-index llms mlx integration
Project description
LlamaIndex Llms Integration: MLX
Overview
Integrate with MLX LLMs from the mlx-lm library
Installation
pip install llama-index-llms-mlx
Example
from llama_index.llms.mlx import MLXLLM
llm = MLXLLM(
model_name="microsoft/phi-2",
tokenizer_name="microsoft/phi-2",
context_window=3900,
max_new_tokens=256,
generate_kwargs={"temp": 0.7, "top_p": 0.95},
)
response = llm.complete("What is the meaning of life?")
print(str(response))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for llama_index_llms_mlx-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1d0e688903748bb7a97c9e0a5dcb7387188a29aea6fcba888dde7a2ad62d479 |
|
MD5 | 1beaf985db6593d94f3bc930a26d8951 |
|
BLAKE2b-256 | 5b65ec148a8ae7b5cb25e12d1b6700479f450404711473d4dfeba8c26087a9bf |
Close
Hashes for llama_index_llms_mlx-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ede5f4fadfc8048193f9f8db4640a7c20992b59219c05efeaa24162e7e74739a |
|
MD5 | 49040709f9b12c4600cb53978b43a4d4 |
|
BLAKE2b-256 | dc04ff6acf61d0c16359e21fcb4bc50019ede3d2d5f788200db07d72ff1d225a |