ASR Adversarial Attacks

This package tests an ASR (speech-to-text) model against well-known white-box adversarial attacks.

Docs: hammaad2002.github.io/ASRAdversarialAttacks

Install

pip install asr-attacks[wav2vec2]

The development version:

pip install "asr-attacks[wav2vec2] @ git+https://github.com/hammaad2002/ASRAdversarialAttacks.git"

Python 3.10+ and PyTorch 2.x are required. Install a CUDA/CPU torch wheel from pytorch.org first if you need a specific build.

Supported attacks

Every attack supports targeted and untargeted modes (untargeted: model prediction or label=). Untargeted Imperceptible is a package extension.

How they behave on a real model (wav2vec2 on LibriSpeech, success rate, distortion and run time) is in the benchmark.

Quickstart

import torch
import torchaudio
from asr_attacks import ASRAttacker, CTCModuleBackend

bundle = torchaudio.pipelines.WAV2VEC2_ASR_BASE_960H
model = bundle.get_model()
device = "cuda" if torch.cuda.is_available() else "cpu"

backend = CTCModuleBackend(model, labels=list(bundle.get_labels()), device=device)
attacker = ASRAttacker(backend)

waveform, sample_rate = torchaudio.load("example.wav")
assert sample_rate == 16000

print("clean:", attacker.decode(waveform))
adv = attacker.fgsm(waveform, epsilon=0.01, targeted=False)
adv = attacker.fgsm(waveform, epsilon=0.01, label="THE CAT", norm=2, targeted=False)
print("adversarial:", attacker.decode(adv))

Targeted BIM:

adv = attacker.bim(
    waveform,
    target="THE CAT SAT",
    epsilon=0.03,
    alpha=0.005,
    num_iter=50,
    targeted=True,
    early_stop=True,
    nested=False,
)
mean_wer, counts = attacker.wer(["THE CAT SAT"], [adv])

Hugging Face CTC:

from asr_attacks import ASRAttacker, HuggingFaceCTCBackend

backend = HuggingFaceCTCBackend("facebook/wav2vec2-base-960h", device="cpu")
attacker = ASRAttacker(backend)

Alternatively, wrap a CTC module with ASRAttacks:

from asr_attacks import ASRAttacks

attacks = ASRAttacks(model, "cpu", list(bundle.get_labels()))
adv = attacks.FGSM_ATTACK(waveform, epsilon=0.01, targeted=False)

Coverage

Tests run on Python 3.10–3.12 (Linux) and 3.12 (macOS, Windows) for every pull request, with branch coverage reported to Codecov.

Responsible use

These methods exist to measure and defend ASR systems. Use them only on models and data you are authorized to evaluate. Do not use this project to interfere with production speech systems you do not own.

Cite

See CITATION.cff. If you use the imperceptible attack, also cite Qin et al. (2019) and IBM ART.

License

Apache License 2.0. Psychoacoustic helpers are derived from the IBM Adversarial Robustness Toolbox; see NOTICE.