audio-xai-fragility 0.0.2

A project about Audio models and it's fragility

Audio XAI Fragility

A project about Audio models and it's fragility

• GitHub | PyPI | Documentation
• Created by Piotr Kitłowski | GitHub @cncPomper | PyPI @pkitlo
• MIT License

Features

• TODO

Documentation

Documentation is built with Zensical and deployed to GitHub Pages.

• Live site: https://cncPomper.github.io/Audio-XAI-Fragility/
• Preview locally: just docs-serve (serves at http://localhost:8000)
• Build: just docs-build

API documentation is auto-generated from docstrings using mkdocstrings.

Docs deploy automatically on push to master via GitHub Actions. To enable this, go to your repo's Settings > Pages and set the source to GitHub Actions.

Development

To set up for local development:

# Clone your fork
git clone [e-mail]:your_username/Audio-XAI-Fragility.git
cd Audio-XAI-Fragility

# Install in editable mode with live updates
uv tool install --editable .

This installs the CLI globally but with live updates - any changes you make to the source code are immediately available when you run audio_xai_fragility.

Run tests:

uv run pytest

Run quality checks (format, lint, type check, test):

just qa

Author

Audio XAI Fragility was created in 2026 by Piotr Kitłowski.

Built with Cookiecutter and the audreyfeldroy/cookiecutter-pypackage project template.

1. General Information and Project Objective

The main objective of the project is to investigate the perceptual fragility of explanations (XAI methods) for deep learning models in the audio domain while keeping predictions unchanged.

2. Planned scope of experiments

• Datasets: Public datasets such as the Speech Commands Dataset (speech) and Sonics (synthetic/real music) will be used. The project will strictly ensure the immutability of the original data.
• Research models: Utilization and adaptation of audio recognition architectures: Audio Spectrogram Transformer, VGGish, Spectra, and ViT.
• XAI methods: Investigation of the vulnerability of gradient-based methods such as Grad-CAM and Integrated Gradients.
• Perceptual constraints: Instead of optimizing attacks against standard metrics, perceptual metrics will be considered (PESQ and STOI for speech, PEAQ for music).
• Computational resources and training: The project will require hardware acceleration (GPUs with a minimum of 16 GB VRAM). The estimated training and fine-tuning time for the base models is approximately 15 hours, while the main process of optimizing perceptual perturbations (XAI attack) for the entire test set is estimated to take an additional 25–30 hours of computation.

3. Planned Program Features

• Classification and Attribution Module: Reading models and generating explanation maps for them.
• Perturbation module: Generating subtle modifications to the audio signal with optimization that preserves high perceptual metrics (e.g., maintaining a PESQ score above 4.0).
• Deployment and Automation: Scripted building, testing, and deployment of applications using tools such as just and Python scripts built with typer or argparse.
• Final deliverables: The project will include clear documentation, user instructions, and tests relevant to the project’s scope.

4. Planned Technology Stack

The project will implement a robust base structure, automatically generated by tools such as cookiecutter or copier.

• Environment management: Use of an isolated virtual environment managed by uv or conda.
• Code cleanliness: Enforced PEP8-compliant coding style with an increased line length limit. Syntax checking provided by an autoformatter (e.g., black or ruff) and a linter (ruff).
• Version control: Rigorous use of a code repository with the conventional commits specification implemented.
• Frameworks and AI: Implementation of learning logic in dedicated frameworks such as PyTorch Lightning in conjunction with Huggingface libraries. Code used for experiments will be continuously exported from Jupyter Lab notebooks into structured library code.
• Experiments and configuration: Tracking progress, metrics, and logs using the Weights & Biases or Tensorboard platform. The configuration of model parameters and experiments will be completely separated from the execution code.
• Documentation: Use of mkdocs to fast and simple write documentation

5. Project schedule

Deadline dates	Planned scope of work and progress
30.03.2026 - 05.04.2026	Repository configuration (Cookiecutter, Ruff, Uv). Defining the directory structure and ensuring that audio files remain immutable.
06.04.2026 - 12.04.2026	Connecting W&B/TensorBoard. Training base classifiers using the PyTorch Lightning framework. (Estimated resource requirements: 15 hours of GPU computation)
13.04.2026 - 19.04.2026	Implementation of explanation-generating (XAI) modules in clean code, after first exporting experiments from notebooks. Writing the first tests.
20.04.2026 - 26.04.2026	Separating configuration from executable code. Preparing baseline attacks on attribution maps using standard distance metrics.
27.04.2026 - 03.05.2026	Implementation of PESQ/STOI/PEAQ metric approximations directly into the attack optimization loop (generation of perceptual perturbations).
04.05.2026 - 10.05.2026	Launch of the main research experiments on a dedicated cluster. (Estimated resource requirements: 25–30 hours of GPU computing for iterative processes).
11.05.2026 - 17.05.2026	Scripting the execution of the entire experiment using the just tool and CLI libraries (e.g., typer). Aggregating tables containing the results.
18.05.2026 - 24.05.2026	Finalization of the work: creating documentation and clear instructions for using the finished system. Organizing the code in accordance with PEP8. Preparation of the paper(?)

---

6. Bibliography Review

Paper	Notes
Interpretation of neural networks is fragile	TODO
Explanations can be manipulated and geometry is to blame	TODO
Constructing adversarial examples to investigate the plausibility of explanations in deep audio and image classifiers	TODO
Perceptual Coding In Python	TODO
TODO	TODO