No project description provided
Project description
mini-judge
Simple implementation of LLM-As-Judge for pairwise evaluation of Q&A models.
Usage
Install the package using pip:
pip install mini-judge
Then, you can use the package as follows.
First, set the OPENAI_API_KEY environment variable to your OpenAI API key.
Then, you can run the following command to evaluate the candidate answers in candidate_answers_path
against the reference answers in ref_answers_path
using judge_model
as the judge model.
mini-judge \
--judge_model <judge_model> \
--questions_path <questions_path> \
--candidate_answers_path <candidate_answers_path> \
--ref_answers_path <ref_answers_path> \
--output_path <output_path>
To run a quick demo, use the following command to evaluate the candidate answers in example_data/candidate_answers.jsonl
against the reference answers in example_data/ref_answers.jsonl
using GPT-4 as the judge model.
mini_judge --output_path <output_path>
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mini_judge-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 67033c285da98a1a55f80f0ce169548969c21b65f21ff191084e6daaaa135d72 |
|
MD5 | 07586c10ca1fa40cb8e866c162651769 |
|
BLAKE2b-256 | 08c8d66a5295caf4f0adcb5cc0a9539408457ec18dcbc11c7b78cb87b1162d47 |