azureml-rag 0.1.3

Contains Retrieval Augmented Generation related utilities for Azure Machine Learning and OSS interoperability.

AzureML Retrieval Augmented Generation Utilities

This package is in alpha stage at the moment, use at risk of breaking changes and unstable behavior.

It contains utilities for:

• Processing text documents into chunks appropriate for use in LLM prompts, with metadata such is source url.
• Embedding chunks with OpenAI or HuggingFace embeddings models, including the ability to update a set of embeddings over time.
• Create MLINdex artifacts from embeddings, a yaml file capturing metadata needed to deserialize different kinds of Vector Indexes for use in langchain. Supported Index types:
  • FAISS index (via langchain)
  • Azure Cognitive Search index

Getting started

You can install AzurrML RAG package via pip.

pip install azureml-rag

Documentation

TBA

Changelog

0.2.0 (2023-05-12)

• Support Workspace Connection based auth for Git, Azure OpenAI and Azure Cognitive Search usage.

0.1.2 (2023-05-05)

• Refactored document chunking to allow insertion of custom processing logic

0.0.1 (2023-04-25)

Features Added

• Introduced package
• langchain Retriever for Azure Cognitive Search