Metrics and visualizations for evaluating chatbot's AI utilization.
Project description
TakeAiEvaluation
TakeAiEvaluation is a tool to provide metrics and visualizations for evaluating a chatbot's AI utilization. This currently addresses two types of evaluation: Knowledge Base Quality and Message Base Information.
Installation
The take_ai_evaluation
package can be installed from PyPI:
pip install take_ai_evaluation
Usage
As input, either a pandas.DataFrame
or a CSV
file path can be used.
- Matrix all vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_all_vs_all_confusion_matrix(title='All vs All')
plt.show()
- Matrix one vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_one_vs_all_confusion_matrix(intent='Intent', title='All vs All')
plt.show()
- Best intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(as_graph=True)
plt.show()
- Worst intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(as_graph=True)
plt.show()
- Classification Report
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report()
- As pandas DataFrame
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report(as_dataframe=True)
Author
Take Blip Data&Analytics Research (ROps)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
take_ai_evaluation-0.2.2.tar.gz
(10.4 kB
view hashes)
Built Distribution
Close
Hashes for take_ai_evaluation-0.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a88777d4dcbbec6a33d502e3b45fb2e14aca4a862abdcaddcaf96ce99d07b7f |
|
MD5 | d45815645a7a42359e83c92bc402cc5f |
|
BLAKE2b-256 | d0a6ea4072b46dad8ad45917af7d0433957eafdb60757171189fe6d3bf11655c |