ferret-xai 0.3.3

Features

ferret builds on top of the transformers library. The library supports explanations using:

• Gradients

• Integrated Gradinets

• Gradient x Input word embeddings

• SHAP

• LIME

and evaluate explanations via:

Faithfulness measures.

• AOPC Comprehensiveness

• AOPC Sufficiency

• Kendall’s tau correlation with leave-one-feature out

Plausibility measures.

• AUPRC soft score plausibility

• Token f1 hard score plausibility

• Token IOU hard score plausibility

TODOs

• Possibility to run on select device (“cpu”, “cuda”)

• Sample-And-Occlusion explanations

• Discretized Integrated Gradients: https://arxiv.org/abs/2108.13654

Visualization

bench = Benchmark(...)

explanations = ...
bench.show_table(explanations)

evaluations = bench.evaluate_explanations(explanations)
bench.show_evaluation_table(evaluations)

Datasets evaluations

bench = Benchmark(...)

hatexdata = bench.load_dataset("hatexplain")
sample_evaluations =  bench.evaluate_samples(hatexdata, [1,2])
bench.show_samples_evaluation_table(sample_evaluations)

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

• Cookiecutter: https://github.com/audreyr/cookiecutter

• audreyr/cookiecutter-pypackage: https://github.com/audreyr/cookiecutter-pypackage

Logo and graphical assets made by Luca Attanasio.

0.1.0 (2022-05-30)

• First release on PyPI.