RAGA AI CATALYST
Project description
RagaAI Catalyst
RagaAI Catalyst is a powerful tool for managing and optimizing LLM projects. It provides functionalities for project management, trace recording, and experiment management, allowing you to fine-tune and evaluate your LLM applications effectively.
Table of Contents
Installation
To install RagaAI Catalyst, you can use pip:
pip install ragaai-catalyst
Configuration
Before using RagaAI Catalyst, you need to set up your credentials. You can do this by setting environment variables or passing them directly to the RagaAICatalyst
class:
from ragaai_catalyst import RagaAICatalyst
catalyst = RagaAICatalyst(
access_key="YOUR_ACCESS_KEY",
secret_key="YOUR_SECRET_KEY",
api_key={"OPENAI_API_KEY": "YOUR_OPENAI_API_KEY"}
)
Usage
Project Management
Create and manage projects using RagaAI Catalyst:
# Create a project
project = catalyst.create_project(
project_name="Test-RAG-App-1",
description="Description of the project"
)
# List projects
projects = catalyst.list_projects()
print(projects)
Trace Management
Record and analyze traces of your RAG application:
from ragaai_catalyst import Tracer
# Start a trace recording
tracer = Tracer(
project_name="Test-RAG-App-1",
metadata={"key1": "value1", "key2": "value2"},
tracer_type="langchain",
pipeline={
"llm_model": "gpt-3.5-turbo",
"vector_store": "faiss",
"embed_model": "text-embedding-ada-002",
}
).start_trace()
# Your code here
# Stop the trace recording
tracer.stop_trace()
# Alternatively, use a context manager
with tracer.trace():
# Your code here
Experiment Management
Create and manage experiments to evaluate your RAG application:
from ragaai_catalyst import Experiment
# Create an experiment
experiment_manager = Experiment(
project_name="Test-RAG-App-1",
experiment_name="Exp-01",
experiment_description="Experiment Description",
dataset_name="Dataset Created from UI",
)
# Add metrics to the experiment
experiment_manager.add_metrics(
metrics={
"hallucination": {"model": "gpt-4o"},
}
)
# Add multiple metrics
experiment_manager.add_metrics(
metrics=[
{"hallucination": {"model": "gpt-4o"}},
{"hallucination": {"model": "gpt-4"}},
{"hallucination": {"model": "gpt-3.5-turbo"}},
]
)
# Get the status of the experiment
status = experiment_manager.get_status()
print("Experiment Status:", status)
# Get the results of the experiment
results = experiment_manager.get_results()
print("Experiment Results:", results)
Dataset Management
Create and manage trace datasets for your projects.
from ragaai_catalyst import Dataset
# Initialize Dataset management for a specific project
dataset_manager = Dataset(project_name="Test-RAG-App-1")
# List existing datasets
datasets = dataset_manager.list_datasets()
print("Exisiting Datasets:", datasets)
# Create a new dataset with filters
dataset_manager.create_dataset(
dataset_name='Test-dataset-1',
filter_list=[
{
"name": "llm_model",
"values": ["gpt-3.5-turbo", "gpt-4"]
},
{
"name": "prompt_length",
"lte": 27,
"gte": 23
}
]
)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ragaai_catalyst-1.0.3b1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 795423d83af7c8023f49f3afb46dfecb63f6b31c2bf1b17251af570f3317978d |
|
MD5 | 3323cf06d58b57d18012de58ddfdc631 |
|
BLAKE2b-256 | 606492f1adfade30f8ecb56177ec77a237a451f53c8ebef80322a31086132c07 |