Skip to main content

Probabilistic Generative Model Programming

Project description

outlinesmlx

outlinesmlx is a minimalistic library aimed at adapting the Outlines library for use with the MLX framework.

To install, use: pip install outlinesmlx.

Outlines provides methods to control the generation of language models to make their output more predictable. Combined with MLX, it enables guided generation with large language models while leveraging Apple Silicon hardware.

Outlines-MLX

Design Principles

We designed it as an adapter that replaces the PyTorch parts of the original Outlines library with MLX compatible components. We will continue to update it actively as Outlines evolves over time.

Versioning

outlinesmlx-x is the MLX adapter for outlines-x. Versions can easily be checked with :

pip list | grep outlines

Why Outlines MLX?

We believe that guided generation is an important technology that will define the future of AI applications beyond chatbots. As Apple Silicon chips become increasingly powerful, we aim to extend guided-generation capabilities to a whole new family of devices. The original Outlines library relies on PyTorch, and adapting it to MLX requires changing many key components.

Installation

outlinesmlx can be installed directly from the PyPI repository:

pip install outlinesmlx

Supported Models

The models are imported using the library mlx-lm.

This allows for seamless importation of quantized models. You can import any model from the HuggingFace hub using this library.

Load Model with an MLX Backend

Refer to the examples folder and the original Outlines library for more use cases.

import outlinesmlx as outlines

model = outlines.models.mlx("mlx-community/Mistral-7B-Instruct-v0.1-4bit-mlx")

prompt = """You are a sentiment-labelling assistant.
Is the following review positive or negative?

Review: This restaurant is just awesome!
"""
answer = outlines.generate.choice(model, ["Positive", "Negative"])(prompt)

Disclaimer

This library is maintained on a monthly basis. Due to the rapid evolution of the MLX framework and the original Outlines library, it may not be up-to-date with their latest advancements. outlinesmlx is designed for experiments with guided generation on Apple Silicon. Please refer to the original Outlines library for an up-to-date implementation.

outlinesmlx is only compatible with MLX models. If you wish to perform guided generation using transformers or other architectures, please use the original Outlines library.

Contributions

We welcome external contributions!

Citation

Please do not forget to cite the original paper:

@article{willard2023efficient,
  title={Efficient Guided Generation for LLMs},
  author={Willard, Brandon T and Louf, R{\'e}mi},
  journal={arXiv preprint arXiv:2307.09702},
  year={2023}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

outlinesmlx-0.0.271.tar.gz (83.1 kB view details)

Uploaded Source

Built Distribution

outlinesmlx-0.0.271-py3-none-any.whl (10.7 kB view details)

Uploaded Python 3

File details

Details for the file outlinesmlx-0.0.271.tar.gz.

File metadata

  • Download URL: outlinesmlx-0.0.271.tar.gz
  • Upload date:
  • Size: 83.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for outlinesmlx-0.0.271.tar.gz
Algorithm Hash digest
SHA256 9580999cf5290c772e72d5fdfecbc0ef113e0042c0535e7250cba3f1c0e5e6a4
MD5 b6a6d423c60326611cddd25dcd385134
BLAKE2b-256 c80dc35281a489f383da5d0c6a1fb026a9110135b69bfb60ecd18f1ff0a94929

See more details on using hashes here.

File details

Details for the file outlinesmlx-0.0.271-py3-none-any.whl.

File metadata

File hashes

Hashes for outlinesmlx-0.0.271-py3-none-any.whl
Algorithm Hash digest
SHA256 d71e80f2186bf49482581a0771e5c256bedc7029d4f5a4877971bc84de7d7864
MD5 875e3c53647f94da1d64c1a2d193d9e3
BLAKE2b-256 f86aadcb5dffdfd731d5fa3ae10060111c85ecbd2fcc9a94ff1d9bc41006f506

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page