ConsistencyBench

Setup

First install the package from PyPI.

pip install consistencybench

Additionally, if you'd like to use the NER metric (consistencybench.metrics.AgreementNER), run the following first.

python -m spacy download en_core_web_sm

Set the arguments from run_eval.py

python run_eval.py --openai_api_key <OPENAI_API_KEY>

Example Output file: consistencybench/result_gpt-3.5-turbo_paraphrasing.csv

Example Jupyter Notebook: example.ipynb

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution for ConsistencyBench 0.1.3
File	Size	Uploaded
consistencybench-0.1.3.tar.gz	15.3 kB	Apr 3, 2024	Details

Table of built distributions (wheels) for ConsistencyBench 0.1.3
File	Interpreter	ABI	Platform	Reset
consistencybench-0.1.3-py3-none-any.whl 19.3 kB Apr 3, 2024	Python 3	none	any	Details

Total release size: 34.6 kB

Download URL	consistencybench-0.1.3.tar.gz
Size	15.3 kB
Tags	Source
SHA-256 checksum How to use checksums	`37058ecb5541537aec8b6f72f0e16417e23cd5c388ef75d2276d4e2dc3974f2c`
BLAKE2b-256 checksum How to use checksums	`d0cea902c4a106ca39be1815100afb2036e576a0092b807b3031e704226f8b14`
Upload date	Apr 3, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.8.2 CPython/3.11.5 Linux/6.8.0-76060800daily20240311-generic`

Download URL	consistencybench-0.1.3-py3-none-any.whl
Size	19.3 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`e331171f8ea81d6da55b55d15409505f58dc1b09d4f331c23c4a56339cb938d8`
BLAKE2b-256 checksum How to use checksums	`d6557ef6798d40d7dfe62e3fea926841ed852b35e5e4cfba3a841354ed286e7d`
Upload date	Apr 3, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.8.2 CPython/3.11.5 Linux/6.8.0-76060800daily20240311-generic`