outlines-haystack

Table of Contents

• 🛠️ Installation
• 📃 Description
• 💻 Usage

🛠️ Installation

pip install outlines-haystack

📃 Description

Outlines is a Python library that allows you to use Large Language Model in a simple and robust way (with structured generation). It is built by .txt.

-- Outlines docs

This library allow you to use outlines generators in your Haystack pipelines!

This library currently supports the following generators:

• JSON: generate a JSON object with a given schema
• Choices: generate text from a list of options. Useful for classification tasks!
• Text: simply generate text
• Regex: ⚠️ coming soon
• Format: ⚠️ coming soon
• Grammar: ⚠️ coming soon

outlines supports a wide range of models and frameworks, we are currently supporting:

• OpenAI/Azure OpenAI
• 🤗 Transformers
• llama-cpp
• mlx-lm

💻 Usage

[!TIP] See the Example Notebooks for complete examples.

All below examples only use the transformers models.

JSON Generation

>>> from pydantic import BaseModel
>>> from outlines_haystack.generators.transformers import TransformersJSONGenerator

>>> class User(BaseModel):
...    name: str
...    last_name: str

>>> generator = TransformersJSONGenerator(
...     model_name="microsoft/Phi-3-mini-4k-instruct",
...     schema_object=User,
...     device="cuda",
... )
>>> generator.warm_up()
>>> generator.run(prompt="Create a user profile with the fields name, last_name")
{'structured_replies': ['{"name": "John", "last_name": "Doe"}']}

Choice Generation

>>> from outlines_haystack.generators.transformers import TransformersChoiceGenerator

>>> generator = TransformersChoiceGenerator(
...     model_name="microsoft/Phi-3-mini-4k-instruct",
...     choices=["Positive", "Negative"],
...     device="cuda",
... )
>>> generator.warm_up()
>>> generator.run(prompt="Classify the following statement: 'I love pizza'")
{'choice': 'Positive'}

Text Generation

[!TIP] While outlines supports classic text generation, it excels at structured generation. For text generation, consider using Haystack's built-in text generators that offer more features.

>>> from outlines_haystack.generators.transformers import TransformersTextGenerator

>>> generator = TransformersTextGenerator(
...     model_name="microsoft/Phi-3-mini-4k-instruct",
...     device="cuda",
... )
>>> generator.warm_up()
>>> generator.run(prompt="What is the capital of Italy?")
{'replies': ['The capital of Italy is Rome.']}

License

outlines-haystack is distributed under the terms of the MIT license.