QA Evaluation
This is a Python library for assessing the quality of question-answering systems, such as systems built with LLM-based agents. It is agnostic to the agent implementation and the LLM it uses.
The evaluation is based on a user-provided reference dataset containing queries, reference responses, and optional reference steps, such as expected tool uses. The evaluator compares these references with the agent's actual responses and executed steps. Reference steps can be grouped to allow some expected steps to occur in any order.
The library provides built-in evaluation metrics and supports user-defined custom metrics (§ Metrics).
Documentation
Maintainers
Developed and maintained by Graphwise. For issues and feature requests, please open a GitHub issue.
License
Apache-2.0 License. See the LICENSE file for details.
1 maintainer
Metadata
1 maintainer
Release files for graphrag-eval 6.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| graphrag_eval-6.4.0.tar.gz | 20.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| graphrag_eval-6.4.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 48.4 kB
Release files / graphrag_eval-6.4.0.tar.gz
| Download URL | graphrag_eval-6.4.0.tar.gz |
|---|---|
| Size | 20.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e4a5e5e1d4130321e04f02fa2f930f725f76f452f619cd0ce027d4afc723d7f0
|
|
BLAKE2b-256 checksum How to use checksums |
9a937942fa24a00c5c8fc76d6c2b7fabea3fd0a845adeaf6ffe5ba222745f797
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.4 CPython/3.12.13 Linux/6.17.0-1018-azure
|
Release files / graphrag_eval-6.4.0-py3-none-any.whl
| Download URL | graphrag_eval-6.4.0-py3-none-any.whl |
|---|---|
| Size | 27.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d1d9b8a8c5c23f6dc922314bde09b0cf8deba3ed08041586fec6fc188dd2a7e3
|
|
BLAKE2b-256 checksum How to use checksums |
759190d3c3d8f30324b3983e65bf3877e7a5d45f01409501ecca25353b70bf8a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.1.4 CPython/3.12.13 Linux/6.17.0-1018-azure
|
