pige 0.0.1

PAI for RAG Evaluation

Pige——PAI for RAG Evaluation

from pige import critic_LLM, critic_Embedding, pige
API_KEY = "your llm api key"
base_url = "your llm server url"
model_name = "your model name"
llm = critic_LLM(api_key=API_KEY, base_url=base_url, model=model_name)
embedding_path = "your local embedding path"
embedding = critic_Embedding(model_path=embedding_path)
metrics = ['faithfulness', 'answer_relevancy', 'context_precision', 'context_recall']
data_samples = {
    'question': ["What is the capital of France?"],
    'contexts': [["Paris is the capital of France."]],
    'answer': ["Paris"],
    'ground_truth': ["Paris"]
}
res = pige(llm, embedding, data_samples, metrics)

Data Format

"question": List[str]

"context": List[List[str]]

"answer": List[str]

"ground_truth": List[str]

Output Format

The output of pige is a dataframe, you can use pandas to process it.