SelfCheckGPT: Assessing text-based responses from LLMs
Project description
Code for our paper “SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models”, https://arxiv.org/abs/2303.08896
More information can be found on our project page: https://github.com/potsawee/selfcheckgpt
Installation
pip install selfcheckgpt
SelfCheckGPT Usage
See more details in Jupyter Notebook: https://github.com/potsawee/selfcheckgpt/blob/main/demo/SelfCheck_demo1.ipynb
from selfcheckgpt.modeling_selfcheck import SelfCheckMQAG, SelfCheckBERTScore
selfcheck_mqag = SelfCheckMQAG()
selfcheck_bertscore = SelfCheckBERTScore()
sent_scores_mqag = selfcheck_mqag.predict(
sentences,
passage,
[sample1, sample2, sample3],
num_questions_per_sent = 5,
scoring_method = 'bayes_with_alpha',
beta1 = 0.8, beta2 = 0.8,
)
sent_scores_bertscore = selfcheck_bertscore.predict(
sentences,
[sample1, sample2, sample3],
)
MQAG Usage
See more details in Jupyter Notebook: https://github.com/potsawee/selfcheckgpt/blob/main/demo/MQAG_demo1.ipynb
from selfcheckgpt.modeling_mqag import MQAG
mqag_model = MQAG()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
selfcheckgpt-0.1.7.tar.gz
(16.7 kB
view details)
File details
Details for the file selfcheckgpt-0.1.7.tar.gz
.
File metadata
- Download URL: selfcheckgpt-0.1.7.tar.gz
- Upload date:
- Size: 16.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38568c38d9398c61976a2739e752e780c759189175f5a1f62c5b58a2d761c706 |
|
MD5 | de19bd437c82e1a122825ad6d298912f |
|
BLAKE2b-256 | f8cbb4c3d96b5df9dfff6bb938d0690011a337880708be20c82f87e4ba7a093d |