Pige——PAI for RAG Evaluation
from pige import critic_LLM, critic_Embedding, pige
API_KEY = "your llm api key"
base_url = "your llm server url"
model_name = "your model name"
llm = critic_LLM(api_key=API_KEY, base_url=base_url, model=model_name)
embedding_path = "your local embedding path"
embedding = critic_Embedding(model_path=embedding_path)
metrics = ['faithfulness', 'answer_relevancy', 'context_precision', 'context_recall']
data_samples = {
'question': ["What is the capital of France?"],
'contexts': [["Paris is the capital of France."]],
'answer': ["Paris"],
'ground_truth': ["Paris"]
}
res = pige(llm, embedding, data_samples, metrics)
Data Format
"question": List[str]
"context": List[List[str]]
"answer": List[str]
"ground_truth": List[str]
Output Format
The output of pige is a dataframe, you can use pandas to process it.
1 maintainer
Metadata
1 maintainer
Release files for pige 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pige-0.0.1.tar.gz | 3.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pige-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 6.0 kB
Release files / pige-0.0.1.tar.gz
| Download URL | pige-0.0.1.tar.gz |
|---|---|
| Size | 3.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b580386b722737a24b797e56ed9fb8b81030c417fad9432696319d50f835b795
|
|
BLAKE2b-256 checksum How to use checksums |
c5d691908c64daa263640a36dff2558acb4ce2e034ce68040d5b9c0f68a23afe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.4
|
Release files / pige-0.0.1-py3-none-any.whl
| Download URL | pige-0.0.1-py3-none-any.whl |
|---|---|
| Size | 3.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
95b0f844511e25798214559c4b6cbdea446495e148085c2a4b131c8bcc73a753
|
|
BLAKE2b-256 checksum How to use checksums |
c0632c1ce7128f844b5f692fdbcf924d17362c82ab1245a06e3cea2b0850e540
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.1 CPython/3.10.4
|
