cafga 0.0.6

CafGa is a library that facilitates creating and evaluating grouped-attribution explanations.

CafGa (Custom assignments for Group attribution)

CafGa is a tool that allows you to create attribution-based explanations at custom granularities. (See the paper for details)

Project Links

Website: https://cafga.ivia.ch/

PyPi package: https://pypi.org/project/cafga/

Installation

CafGa can be installed through PyPI using

pip install cafga

When running CafGa from the repository run:

pip install -r requirements.txt

Note that some of the extra functionality requires further installations:

1. CafGa provides two jupyter widgets. The edit widget allows one to visually edit assignments and the display widget displays the attributions generated by the explanation. The widgets require additional javascript packages install via npm. Follow the instructions in the 'Demo Instructions.md' file to download the packages.

2. CafGa offers a predefined ChatGPT model. To use it you need to add your API key to the environment variables. To do so you can place a .env file in your working directory (from where you run your python file or notebook) with the following content:

OPENAI_API_KEY=your_api_key_here

Then you need to load the .env file using the python package python-dotenv. You can do so by adding the following lines to your code before instantiating the CafGa object:

from dotenv import load_dotenv
load_dotenv()

In case this does not work you can try placing the .env file in the cafga package directory. If you are using a environment manager like conda the path should look like this: *path_to_envs_directory*/*environment name*/lib/python*version number*/site-packages/cafga

Using CafGa

The following provides an explanation of the main functions of cafga. The code shown in the examples is also provided in the demo.ipynb notebook. To use widgets described in the demo please follow the instructions in the 'Demo Instructions.md' file to set up the necessary javascript packages.

To begin using CafGa, start by importing CafGa creating a cafga object:

from cafga import CafGa

cafga = CafGa(model = 'your_model')

Example (using the predefined chatgpt model)

# These first three commands are for the widgets
%load_ext autoreload
%autoreload 2
%env ANYWIDGET_HMR=1

from cafga import CafGa

cafga = CafGa(model="chatgpt")

The model parameter is where you pass the model you want to explain. To allow for parallelization in how your model generates predictions (e.g. by batching) cafga sends lists of inputs to your model instead of single inputs. Thus, the function that implements your model should take a list of strings as input and output either a list of strings or a list of floats as output (i.e. a list containing one output for every input).

Once cafga is instantiated the typical usage of cafga runs proceeds in three steps: Explanation, Evaluation, and Visualisation.

1. Explanation

To generate an explanation run the explain function on the instantiated cafga object:

explanation = cafga.explain(args)

Example (using the widgets to define the explanation inputs)

We first define the task (the template and target answer are optional)

# Example Task
input_text = """I'm excited to use CafGa, but I'm not sure about it's usage."""
template = """For the text snippet below answer whether it has positive or negative sentiment. 
Your answer should be only one word: 'posiitive' or 'negative'. Text snippet:\n
{input}
"""
target_answer = "positive"

Now we use the edit widget to define the custom group assignments for the explanation. Here we use word-level segmentation as the text segmentation. For your own use case you do not need to use the widget. You can directly provide the segmented input and group assignments to the explain function as described further below. You can also try some of the predefined assignments methods provided in cafga using the "assignment method" argument.

cafga.edit_assignments(input_text, "word")

# When you press confirm in the widget above you can see the updated input below
edited_input = cafga.get_edited_input()

Here you can see the resulting segmentation and group assignments we get from the edit widget. Note the format of the input segments and group assignments; Here the words ["I'm ", 'to ', 'use ', 'CafGa, '] belong to group 0, ['excited '] belongs to group 1, and ['but ', "I'm ", 'not ', 'sure ','about ', "it's ", 'usage.'] belong to group 2. The selected evaluation method will be relevant in the next step.

Sample: example
With input segments: ["I'm ", 'excited ', 'to ', 'use ', 'CafGa, ', 'but ', "I'm ", 'not ', 'sure ', 'about ', "it's ", 'usage.']
Assigned as: [0, 1, 0, 0, 0, 2, 2, 2, 2, 2, 2, 2]
To be evaluated with: deletion

Now we have all the ingredients to generate the explanation:

1. The segmented input: The atomic units of the input text. For example, tokens or words.
2. The custom group assignments: The assignment of each input segment to a group.
3. The template (optional): A string template that describes the task to the model. This is used for generative models. The template should contain the placeholder {input} which will be replaced by the input text.
4. The scalarizer (optional): A function that maps the model's output to a scalar value. (This is used for generative models that return text as output. It is not necessary if your model already outputs a scalar value.)

scalarizer = [("EQUAL", target_answer), ("CONTAIN", target_answer)]
explanation = cafga.explain(
    segmented_input=edited_input.input_segments,
    custom_assignments=edited_input.group_assignments,
    template=template,
    scalarizer=scalarizer,
)

We can now use the display widget to visualise the generated explanation. Since cafga supports using multiple scalarizers at once we need to specify for scalarizer we want to display attributions. Here we display the attributions generated from the first scalarizer (index 0).

scalarizer_index = 0
cafga.display_explanation(
    input_segments=edited_input.input_segments,
    group_assignments=edited_input.group_assignments,
    attributions=explanation.attributions[scalarizer_index],
    sample_name=edited_input.sample_name,
)

2. Evaluation

Once an explanation object has been generated you can pass it on to the evaluation function:

evaluation = cafga.evaluate(explanation, args)

The two forms of evaluation currently supported are deletion (going from all features present to no features present) and insertion (going from no features present to all features present), which can be indicated by the direction parameter. The resulting evaluation accordinlgy contains the array of difference values computed as part of the perturbation curve.

Example: We take the explanation generated from cafga.explain and evaluate it with deletion which we specificed in the edit widget above.

evaluation = cafga.evaluate(
    explanation,
    scalarizer=scalarizer,
    direction=edited_input.direction,
)

3. Visualisation

Finally, the perturbation curve generated by the evaluation can be visualised using the visualisation function:

cafga.visualize_evaluation(evaluated_explanations, args)

Since you may want to plot the aggregate over many evaluations the visualisation functions takes in a list of evaluations as input. The two forms of aggregation currently supported are equal width binning and linear interpolation.

Example: We visualise the evaluation generated in the previous step.

cafga.visualize_evaluation([evaluation], scalarizer_index=scalarizer_index)

Citation

The paper can be found here. Please cite CafGa if you use it in your work:

Bibtext:

@inproceedings{boyle2025cafga,
    title={CafGa: Customizing Feature Attributions to Explain Language Models}, 
    author={Alan Boyle and Furui Cheng and Vilém Zouhar and Mennatallah El-Assady},
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    year={2025},
    month={Nov},
    address={Suzhou, China},
    publisher={Association for Computational Linguistics},
    url={https://arxiv.org/abs/2509.20901}, 
}

Pre-formatted:

Alan Boyle, Furui Cheng, Vilém Zouhar, and Mennatallah El-Assady. 2025. "CafGa: Customizing Feature Attributions to Explain Language Models.". In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Suzhou, China. Association for Computational Linguistics.