outlines-haystack 0.1.0

Use `outlines` generators with Haystack.

outlines-haystack

---

Table of Contents

• 🛠️ Installation
• 📃 Description
• 💻 Usage

🛠️ Installation

pip install outlines-haystack

📃 Description

Outlines is a Python library that allows you to use Large Language Model in a simple and robust way (with structured generation). It is built by .txt.

-- Outlines docs

This library allow you to use outlines generators in your Haystack pipelines!

This library currently supports the following generators:

• JSON: generate a JSON object with a given schema
• Text: simply generate text
• Choices: ⚠️ coming soon
• Regex: ⚠️ coming soon
• Format: ⚠️ coming soon
• Grammar: ⚠️ coming soon

outlines supports a wide range of models and frameworks, we are currently supporting:

• OpenAI/Azure OpenAI
• 🤗 Transformers
• mlx-lm

💻 Usage

[!TIP] See the Example Notebooks for complete examples.

All below examples only use the transformers models.

JSON Generation

>>> from enum import Enum
>>> from pydantic import BaseModel
>>> from outlines_haystack.generators.transformers import TransformersJSONGenerator
>>> class User(BaseModel):
...    name: str
...    last_name: str

>>> generator = TransformersJSONGenerator(
...     model_name="microsoft/Phi-3-mini-4k-instruct",
...     schema_object=User,
...     device="cuda",
...     sampling_algorithm_kwargs={"temperature": 0.5},
... )
>>> generator.warm_up()
>>> generator.run(prompt="Create a user profile with the fields name, last_name")
{'structured_replies': [{'name': 'John', 'last_name': 'Doe'}]}

Text Generation

>>> generator = TransformersTextGenerator(
...     model_name="microsoft/Phi-3-mini-4k-instruct",
...     device="cuda",
...     sampling_algorithm_kwargs={"temperature": 0.5},
... )
>>> generator.warm_up()
>>> generator.run(prompt="What is the capital of Italy?")
{'replies': ['\n\n# Answer\nThe capital of Italy is Rome.']}

License

outlines-haystack is distributed under the terms of the MIT license.