ConsistencyBench
Setup
First install the package from PyPI.
pip install consistencybench
Additionally, if you'd like to use the NER metric (consistencybench.metrics.AgreementNER), run the following first.
python -m spacy download en_core_web_sm
To start generation and scoring
Set the arguments from run_eval.py
python run_eval.py --openai_api_key <OPENAI_API_KEY>
Example Output file: consistencybench/result_gpt-3.5-turbo_paraphrasing.csv
Example Jupyter Notebook: example.ipynb
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
consistencybench-0.1.3.tar.gz
(15.3 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file consistencybench-0.1.3.tar.gz.
File metadata
- Download URL: consistencybench-0.1.3.tar.gz
- Upload date:
- Size: 15.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.8.2 CPython/3.11.5 Linux/6.8.0-76060800daily20240311-generic
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
37058ecb5541537aec8b6f72f0e16417e23cd5c388ef75d2276d4e2dc3974f2c
|
|
| MD5 |
26dcfa252835572e5a2d8f7a1b998141
|
|
| BLAKE2b-256 |
d0cea902c4a106ca39be1815100afb2036e576a0092b807b3031e704226f8b14
|
File details
Details for the file consistencybench-0.1.3-py3-none-any.whl.
File metadata
- Download URL: consistencybench-0.1.3-py3-none-any.whl
- Upload date:
- Size: 19.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.8.2 CPython/3.11.5 Linux/6.8.0-76060800daily20240311-generic
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e331171f8ea81d6da55b55d15409505f58dc1b09d4f331c23c4a56339cb938d8
|
|
| MD5 |
fc404981cfbd1fa902ce8730a8f2aac8
|
|
| BLAKE2b-256 |
d6557ef6798d40d7dfe62e3fea926841ed852b35e5e4cfba3a841354ed286e7d
|
