Metrics and visualizations for evaluating chatbot's AI utilization.
Project description
TakeAiEvaluation
TakeAiEvaluation is a tool to provide metrics and visualizations for evaluating a chatbot's AI utilization. This currently addresses two types of evaluation: Knowledge Base Quality and Message Base Information.
Installation
The take_ai_evaluation
package can be installed from PyPI:
pip install take_ai_evaluation
Usage
As input, either a pandas.DataFrame
or a CSV
file path can be used.
- Matrix all vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_all_vs_all_confusion_matrix(title='All vs All')
plt.show()
- Matrix one vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_one_vs_all_confusion_matrix(intent='Intent', title='All vs All')
plt.show()
- Best intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(as_graph=True)
plt.show()
- Worst intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(as_graph=True)
plt.show()
Author
Take Blip Data&Analytics Research (ROps)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for take_ai_evaluation-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3d64abed0c6ca71106dd96cd14af5da8c1e81823d87e3dc068f1acee344e9bd |
|
MD5 | 0ba693375cbc476ffd44b0bccabb491d |
|
BLAKE2b-256 | fc150d4711a21ddb03d6f63ffd3893414f3ab060a4797c7dbad805a851159811 |