Metrics and visualizations for evaluating chatbot's AI utilization.
Project description
TakeAiEvaluation
TakeAiEvaluation is a tool to provide metrics and visualizations for evaluating a chatbot's AI utilization. This currently addresses two types of evaluation: Knowledge Base Quality and Message Base Information.
Installation
The take_ai_evaluation
package can be installed from PyPI:
pip install take_ai_evaluation
Usage
As input, either a pandas.DataFrame
or a CSV
file path can be used.
- Matrix all vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_all_vs_all_confusion_matrix(title='All vs All')
plt.show()
- Matrix one vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_one_vs_all_confusion_matrix(intent='Intent', title='All vs All')
plt.show()
- Best intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(as_graph=True)
plt.show()
- Worst intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(as_graph=True)
plt.show()
- Classification Report
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report()
- As pandas DataFrame
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report(as_dataframe=True)
Author
Take Blip Data&Analytics Research (ROps)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
take_ai_evaluation-0.2.3.tar.gz
(11.0 kB
view hashes)
Built Distribution
Close
Hashes for take_ai_evaluation-0.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c337046818fd80f55a56ef33058c1481d0659bc4315b331cada2743e978dd748 |
|
MD5 | 15005dfdd2a51228ba74f9bde03e85b0 |
|
BLAKE2b-256 | cbadc13dc5c836b46cc7b81c258190ffda1fbff351960658b2575da467ae6f54 |