Probabilistic Generative Model Programming
Project description
outlinesmlx
outlinesmlx
is a minimalistic library aimed at adapting the Outlines library for use with the MLX framework.
To install, use: pip install outlinesmlx
.
Outlines provides methods to control the generation of language models to make their output more predictable. Combined with MLX, it enables guided generation with large language models while leveraging Apple Silicon hardware.
Design Principles
We designed it as an adapter that replaces the PyTorch parts of the original Outlines library with MLX compatible components. We will continue to update it actively as Outlines evolves over time.
Versioning
outlinesmlx-x
is the MLX adapter for outlines-x
. Versions can easily be checked with :
pip list | grep outlines
Why Outlines MLX?
We believe that guided generation is an important technology that will define the future of AI applications beyond chatbots. As Apple Silicon chips become increasingly powerful, we aim to extend guided-generation capabilities to a whole new family of devices. The original Outlines library relies on PyTorch, and adapting it to MLX requires changing many key components.
Installation
outlinesmlx
can be installed directly from the PyPI repository:
pip install outlinesmlx
Supported Models
The models are imported using the library mlx-lm.
This allows for seamless importation of quantized models. You can import any model from the HuggingFace hub using this library.
Load Model with an MLX Backend
Refer to the examples folder and the original Outlines library for more use cases.
import outlinesmlx as outlines
model = outlines.models.mlx("mlx-community/Mistral-7B-Instruct-v0.1-4bit-mlx")
prompt = """You are a sentiment-labelling assistant.
Is the following review positive or negative?
Review: This restaurant is just awesome!
"""
answer = outlines.generate.choice(model, ["Positive", "Negative"])(prompt)
Disclaimer
This library is maintained on a monthly basis. Due to the rapid evolution of the MLX framework and the original Outlines library, it may not be up-to-date with their latest advancements. outlinesmlx
is designed for experiments with guided generation on Apple Silicon. Please refer to the original Outlines library for an up-to-date implementation.
outlinesmlx
is only compatible with MLX models. If you wish to perform guided generation using transformers or other architectures, please use the original Outlines library.
Contributions
We welcome external contributions!
Citation
Please do not forget to cite the original paper:
@article{willard2023efficient,
title={Efficient Guided Generation for LLMs},
author={Willard, Brandon T and Louf, R{\'e}mi},
journal={arXiv preprint arXiv:2307.09702},
year={2023}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file outlinesmlx-0.0.271.tar.gz
.
File metadata
- Download URL: outlinesmlx-0.0.271.tar.gz
- Upload date:
- Size: 83.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9580999cf5290c772e72d5fdfecbc0ef113e0042c0535e7250cba3f1c0e5e6a4 |
|
MD5 | b6a6d423c60326611cddd25dcd385134 |
|
BLAKE2b-256 | c80dc35281a489f383da5d0c6a1fb026a9110135b69bfb60ecd18f1ff0a94929 |
File details
Details for the file outlinesmlx-0.0.271-py3-none-any.whl
.
File metadata
- Download URL: outlinesmlx-0.0.271-py3-none-any.whl
- Upload date:
- Size: 10.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d71e80f2186bf49482581a0771e5c256bedc7029d4f5a4877971bc84de7d7864 |
|
MD5 | 875e3c53647f94da1d64c1a2d193d9e3 |
|
BLAKE2b-256 | f86aadcb5dffdfd731d5fa3ae10060111c85ecbd2fcc9a94ff1d9bc41006f506 |