Measuring Biases in Masked Language Models for PyTorch Transformers

Evaluate biases in pre-trained or re-trained masked language models (MLMs), such as those available through HuggingFace. This package computes bias scores across various bias types, using benchmark datasets like CrowS-Pairs (CPS) and StereoSet (SS) (intrasentence), or custom datasets. You can also compare relative bias between two MLMs, or evaluate re-trained MLMs versus their pre-trained base models.

Evaluation Methods

Bias scores for an MLM are computed for sentence pairs in the dataset using measures that represent MLM preference (or prediction quality). Bias against disadvantaged groups for a sentence pair is represented by a higher relative measure value for a sentence in adv compared to dis.

Iterative Masking Experiment (IME): For each sentence, an MLM masks one token at a time until all tokens are masked once, generating n logits or predictions for a sentence with n tokens.

Measures

We use state-of-the-art measures computed under the IME:

• CRR: Difference in reciprocal rank of a predicted token (always equal to 1) and the reciprocal rank of a masked token arXiv
• CRRA: CRR with Attention weights arXiv
• ΔP: Difference in log-liklihood of a predicted token and the masked token arXiv
• ΔPA: ΔP with Attention weights arXiv

Pseudo log-likelihood measures computed with a single encoded input (see References for more details):

• CSPS: CrowS-Pairs Score is a score for an MLM selecting unmodified tokens given modified ones arXiv
• SSS: StereoSet Score is a score for an MLM selecting modified tokens given unmodified ones arXiv
• AUL: All Unmasked Likelihood is a score generated by predicting all tokens in a single unmasked input arXiv
• AULA: AUL with Attention weights arXiv

Setup

pip install mlm-bias

import mlm_bias

# Load and sample the CPS benchmark dataset
cps_dataset = mlm_bias.BiasBenchmarkDataset("cps")
cps_dataset.sample(indices=list(range(10)))

# Specify the model
model = "bert-base-uncased"

# Initialize the BiasMLM evaluator
mlm_bias_evaluator = mlm_bias.BiasMLM(model, cps_dataset)

# Evaluate the model (can pass batching=True for larger samples)
result = mlm_bias_evaluator.evaluate(attention=True)

# Print the results, including total bias scores and scores by category for each measure
def format_mlm_bias_results(result, total_only=False, sep="\n"):
    outl = []
    if total_only:
        for measure in result['bias_scores'].keys():
            outl.append((f"{measure.replace('d','Δ').upper()} " \
                         f"total = {round(result['bias_scores'][measure]['total'],3)}"))
    else:
        for measure in result['bias_scores'].keys():
            outl.append(f"Measure = {measure.replace('d','Δ').upper()}")
            for bias_type, score in result['bias_scores'][measure].items():
                outl.append(f"- {bias_type} = {round(score,3)}")
    return f"{sep}".join(outl)
print(format_mlm_bias_results(result))

# Save the results
result.save("./mlm-bias_cps_bert-base-uncased")

Example Script

Clone the repository and install the package:

git clone https://github.com/zalkikar/mlm-bias.git
cd mlm-bias
python3 -m pip install .

Run the mlm_bias.py example script:

usage: mlm_bias.py [-h] --data {cps,ss,custom} --model_name_or_path MODEL_NAME_OR_PATH [--model_name_or_path_2 MODEL_NAME_OR_PATH_2]
                   [--output OUTPUT] [--measures {all,crr,crra,dp,dpa,aul,aula,csps,sss}] [--start START] [--end END] [--batching]

options:
  -h, --help            show this help message and exit
  --data {cps,ss,custom}
                        Paired sentences from benchmark or supplied line by line dataset in /data directory. Provide bias types in
                        "<data>/bias_types.txt" and biased sentences in "<data>/dis.txt" and "<data>/adv.txt" accordingly.
  --model_name_or_path MODEL_NAME_OR_PATH
                        Model (MLM) to compute bias measures for. Must be supported by HuggingFace.
  --model_name_or_path_2 MODEL_NAME_OR_PATH_2
                        Model (MLM) to compute bias measures for. Must be supported by HuggingFace. Used to compare with "--model"
  --output OUTPUT       Full path (eg. dir/file.txt) for output directory with computed measures.
  --measures {all,crr,crra,dp,dpa,aul,aula,csps,sss}
                        Measures computed to evaluate bias in MLMs.
  --start START         Start index of dataset sample.
  --end END             End index of dataset sample.
  --batching            Batched inputs.

Example arguments:

# Single MLM
python3 mlm_bias.py --data cps --model_name_or_path roberta-base --start 0 --end 30 --batching
python3 mlm_bias.py --data ss --model_name_or_path bert-base-uncased --start 0 --end 30 --batching

# Relative between two MLMs
python3 mlm_bias.py --data cps --model_name_or_path roberta-base --start 0 --end 30 --model_name_or_path_2 bert-base-uncased --batching

Output directories (default arguments):

• /data contains cps.csv (CPS) and/or ss.csv (SS).
• /eval contains out.txt with computed bias scores and pickled result objects.

Example Output:

python3 mlm_bias.py --data cps --model_name_or_path bert-base-uncased --start 0 --end 30 --batching

Created output directory.
Created Data Directory |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s
Downloaded Data [CrowSPairs] |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s
Loaded Data [CrowSPairs] |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s
Evaluating Bias [bert-base-uncased] |██████████████████████████████| 30/30 [100%] in 31s ETA: 0s
Saved bias results for bert-base-uncased in ./eval/bert-base-uncased
Saved scores in ./eval/out.txt
--------------------------------------------------
MLM: bert-base-uncased
CRR total = 76.667
CRRA total = 60.0
ΔP total = 63.333
ΔPA total = 63.333
AUL total = 63.333
AULA total = 66.667
SSS total = 50.0
CSPS total = 63.333

In this example, ./eval/out.txt also contains bias scores by category for each measure.

Custom Datasets

Compute bias scores for a custom dataset directory with the following line-by-line files:

• bias_types.txt containing bias categories.
• dis.txt and adv.txt containing sentence pairs, where:
  • dis.txt contains sentences with bias against disadvantaged groups (stereotypical) and
  • adv.txt contains sentences with bias against advantaged groups (anti-stereotypical).

Citation

If using this for research, please cite the following:

@misc{zalkikar2024measuringsocialbiasesmasked,
      title={Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality},
      author={Rahul Zalkikar and Kanchan Chandra},
      year={2024},
      eprint={2402.13954},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2402.13954}
}

References

@article{Kaneko_Bollegala_2022,
      title={Unmasking the Mask – Evaluating Social Biases in Masked Language Models},
      volume={36},
      url={https://ojs.aaai.org/index.php/AAAI/article/view/21453},
      DOI={10.1609/aaai.v36i11.21453},
      number={11},
      journal={Proceedings of the AAAI Conference on Artificial Intelligence},
      author={Kaneko, Masahiro and Bollegala, Danushka},
      year={2022},
      month={Jun.},
      pages={11954-11962}
}

@InProceedings{10.1007/978-3-031-33374-3_42,
      author="Salutari, Flavia
        and Ramos, Jerome
        and Rahmani, Hossein A.
        and Linguaglossa, Leonardo
        and Lipani, Aldo",
      editor="Kashima, Hisashi
        and Ide, Tsuyoshi
        and Peng, Wen-Chih",
      title="Quantifying the Bias of Transformer-Based Language Models for African American English in Masked Language Modeling",
      booktitle="Advances in Knowledge Discovery and Data Mining",
      year="2023",
      publisher="Springer Nature Switzerland",
      address="Cham",
      pages="532--543",
      isbn="978-3-031-33374-3"
}

@inproceedings{nangia-etal-2020-crows,
      title = "{C}row{S}-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models",
      author = "Nangia, Nikita  and
        Vania, Clara  and
        Bhalerao, Rasika  and
        Bowman, Samuel R.",
      editor = "Webber, Bonnie  and
        Cohn, Trevor  and
        He, Yulan  and
        Liu, Yang",
      booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
      month = nov,
      year = "2020",
      address = "Online",
      publisher = "Association for Computational Linguistics",
      url = "https://aclanthology.org/2020.emnlp-main.154",
      doi = "10.18653/v1/2020.emnlp-main.154",
      pages = "1953--1967"
}

@inproceedings{nadeem-etal-2021-stereoset,
      title = "{S}tereo{S}et: Measuring stereotypical bias in pretrained language models",
      author = "Nadeem, Moin  and
        Bethke, Anna  and
        Reddy, Siva",
      editor = "Zong, Chengqing  and
        Xia, Fei  and
        Li, Wenjie  and
        Navigli, Roberto",
      booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
      month = aug,
      year = "2021",
      address = "Online",
      publisher = "Association for Computational Linguistics",
      url = "https://aclanthology.org/2021.acl-long.416",
      doi = "10.18653/v1/2021.acl-long.416",
      pages = "5356--5371"
}