lit-nlp 0.1rc2

Language Interpretability Tool.

Language Interpretability Tool (LIT)

The Language Interpretability Tool (LIT) is a visual, interactive model-understanding tool for NLP models.

LIT is built to answer questions such as:

• What kind of examples does my model perform poorly on?
• Why did my model make this prediction? Can this prediction be attributed to adversarial behavior, or to undesirable priors in the training set?
• Does my model behave consistently if I change things like textual style, verb tense, or pronoun gender?

LIT supports a variety of debugging workflows through a browser-based UI. Features include:

• Local explanations via salience maps, attention, and rich visualization of model predictions.
• Aggregate analysis including custom metrics, slicing and binning, and visualization of embedding spaces.
• Counterfactual generation via manual edits or generator plug-ins to dynamically create and evaluate new examples.
• Side-by-side mode to compare two or more models, or one model on a pair of examples.
• Highly extensible to new model types, including classification, regression, span labeling, seq2seq, and language modeling. Supports multi-head models and multiple input features out of the box.
• Framework-agnostic and compatible with TensorFlow, PyTorch, and more.

The source code and documentation can be found at https://github.com/pair-code/lit.