Inspeq AI SDK
Project description
Inspeqai python SDK
- Website: https://www.inspeq.ai
- Inspeq app: https://app.inspeq.ai
- Detailed Documentation: https://docs.inspeq.ai
Quikstart
Create a Virtual Environment in Linux and Windows
Linux OS / MAC OS
Using venv (Python 3)
- Open a terminal.
- Navigate to the directory where you want to create the virtual environment.
- Run the following command:
python3 -m venv venv
Activate it
source venv/bin/activate
windows
- Open a terminal.
- Navigate to the directory where you want to create the virtual environment.
- Run the following command:
python -m venv venv
Activate it
venv\Scripts\activate
Make sure your environment is activated everytime you use package
SDK Installation
Enter below Command in terminal
pip install inspeqai
Get SDK API keys
Get your API keys from Here
Usage
Create main.py and you can use below code snippet
from inspeq.client import Evaluator
#initialization
API_KEY = "your_sdk_api_key"
inspeq_instance = Evaluator(sdk_api_key=API_KEY)
# Example input data
input_data={
"prompt":"llm_prompt",
"response":" llm_output "
}
'''Note : Do not change the structure of input data keep the structure as it
is. Put your data at places of llm_prompt, llm_output
and your_llm_output .
'''
print("Word limit test :", inspeq_instance.word_limit_test(input_data))
Get all metrics
from inspeq.client import Evaluator
#initialization
API_KEY = "your_sdk_api_key"
inspeq_instance = Evaluator(sdk_api_key=API_KEY)
# Example input data
# three parameters are required for get_all_metrics you can see below ,do not change structure inside the input data
input_data={
"prompt":"your_llm_prompt",
"context":"your_llm_context",
"response":"your_llm_output "
}
'''Note : Do not change the structure of input data keep the structure as it
is you need to include prompt,context,response as it is . Put your data at places of your_llm_prompt, your_llm_context
and your_llm_output .
'''
#get all metrics in one function
print(inspeq_instance.get_all_metrics(input_data))
After you run the file all metrics result will print in your terminal or output window.
All Metrics provided by Inspeq sdk
Different metrics required different parameters you can visit official documentation
Click Here
print("Factual Consistency:", inspeq_instance.factual_consistency(input_data))
print("Answer Relevance:", inspeq_instance.answer_relevance(input_data))
print("Response Tone:", inspeq_instance.response_tone(input_data))
print("Grammatical Correctness:", inspeq_instance.grammatical_correctness(input_data))
print("Fluency:", inspeq_instance.fluency(input_data))
print("Do Not Use Keywords:", inspeq_instance.do_not_use_keywords(input_data))
print("Word Limit Test:", inspeq_instance.word_limit_test(input_data))
print("Conceptual Similarity:", inspeq_instance.conceptual_similarity(input_data))
print("Coherence:", inspeq_instance.coherence(input_data))
print("Readability:", inspeq_instance.readability(input_data))
print("Clarity:", inspeq_instance.clarity(input_data))
print("Get all metrics:", inspeq_instance.get_all_metrics(input_data))
Supported Features
Metrices:
-
Factual Consistency: Factual Consistency (FC) pertains to the precision and correctness of information articulated in text produced by Large Language Models (LLMs). It involves the comparison of generated information with the given context, input, or anticipated factual knowledge.
-
Grammatical Correctness: Assess the grammatical accuracy of the generated text.
-
Do Not Use Keywords: Identify and evaluate the use of specific keywords or phrases.
-
Answer Fluency: Fluency refers to the ability of the LLM to generate text that is grammatically correct, natural-sounding, and easy to read.
-
Answer Relevance: Determine the relevance of the generated text in the context of a given query or
-
Word Limit Test: Check if the generated text adheres to specified word limits.
-
Response Tonality: Tonality refers to the type of tone or overall sentiment highlighted in the response.
-
Conceptual Similarity: This refers to the semantic similarity or relatedness between response generated and provided context.
-
Coherence: Coherence metric evaluates the ability of the LLM to generate text that is organized, well-structured, and easy to understand.
-
Readibility: Readability scores help assess whether the LLM’s generated text is appropriate for the target audience’s reading level.
-
Clarity: Clarity is a subjective metric and refers to the response’s clarity in terms of language and structure.
-
Get_all_metrics: This will provide result of all metrics.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.