Metrics and visualizations for evaluating chatbot's AI utilization.
Project description
TakeAiEvaluation
TakeAiEvaluation is a tool to provide metrics and visualizations for evaluating a chatbot's AI utilization. This currently addresses two types of evaluation: Knowledge Base Quality and Message Base Information.
Installation
The take_ai_evaluation
package can be installed from PyPI:
pip install take_ai_evaluation
Usage
As input, either a pandas.DataFrame
or a CSV
file path can be used.
- Matrix all vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_all_vs_all_confusion_matrix(title='All vs All')
plt.show()
- Matrix one vs all
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_one_vs_all_confusion_matrix(intent='Intent', title='All vs All')
plt.show()
- Best intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_best_intent(as_graph=True)
plt.show()
- Worst intent
- Just the values for the default metric, which is 'accuracy'
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent()
plt.show()
- Just the values for 'recall' metric
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(metric='recall')
plt.show()
- As graph
import matplotlib.pyplot as plt
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_worst_intent(as_graph=True)
plt.show()
- Classification Report
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report()
- As pandas DataFrame
from take_ai_evaluation import AiEvaluation
ai_evaluation = AiEvaluation(analysed_base='knowledge-base.csv',
sentence_col='id',
intent_col='intent',
predict_col='predicted')
ai_evaluation.get_classification_report(as_dataframe=True)
Author
Take Blip Data&Analytics Research (ROps)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
take_ai_evaluation-0.2.1.tar.gz
(10.4 kB
view hashes)
Built Distribution
Close
Hashes for take_ai_evaluation-0.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e01e5629bc31a7f0dae036c4bc4854a69607456df30a4cbdd41280f6c6f92bca |
|
MD5 | 593b294866fb12b57719740238698dc0 |
|
BLAKE2b-256 | 83318b8af551c9eb0c58ea90a42702a73c52ce9826de253352f8dfe4d3b546cf |