Audio XAI

XAI for audio models

• GitHub | PyPI | Documentation
• Created by Piotr Kitłowski | GitHub @cncPomper | PyPI @pkitlo
• MIT License

Features

• TODO

Documentation

Documentation is built with Zensical and deployed to GitHub Pages.

• Live site: https://cncPomper.github.io/Audio-XAI/
• Preview locally: just docs-serve (serves at http://localhost:8000)
• Build: just docs-build

API documentation is auto-generated from docstrings using mkdocstrings.

Docs deploy automatically on push to master via GitHub Actions. To enable this, go to your repo's Settings > Pages and set the source to GitHub Actions.

Setup

If you don't have ffmpeg installed, run:

Windows 11

winget install -e --id Gyan.FFmpeg

if you use conda, thats should enough

conda install -c conda-forge ffmpeg -y

Then verify the installation:

ffmpeg -version
ffprobe -version

Running tensorboard

tensorboard --logdir Audio-XAI/runs/ --port 6006 --bind_all

Examples

https://drive.google.com/drive/folders/1XD1rJwqSX-y-2OHzudYaatKE92MlYY8_?usp=sharing

Development

To set up for local development:

# Clone your fork
git clone git@github.com:your_username/Audio-XAI.git
cd Audio-XAI

# Install in editable mode with live updates
uv tool install --editable .

This installs the CLI globally but with live updates - any changes you make to the source code are immediately available when you run audio_xai.

Run tests:

uv run pytest

Run quality checks (format, lint, type check, test):

just qa

Author

Audio XAI was created in 2026 by Piotr Kitłowski.

Built with Cookiecutter and the audreyfeldroy/cookiecutter-pypackage project template.

1. General Information and Project Objective

The main objective of the project is to investigate the perceptual fragility of explanations (XAI methods) for deep learning models in the audio domain while keeping predictions unchanged.

2. Planned scope of experiments

• Datasets: Public datasets such as the Speech Commands Dataset (speech) and Sonics (synthetic/real music) will be used. The project will strictly ensure the immutability of the original data.
• Research models: Utilization and adaptation of audio recognition architectures: Audio Spectrogram Transformer, VGGish, Spectra, and ViT.
• XAI methods: Investigation of the vulnerability of gradient-based methods such as Grad-CAM and Integrated Gradients.
• Perceptual constraints: Instead of optimizing attacks against standard metrics, perceptual metrics will be considered (PESQ and STOI for speech, PEAQ for music).
• Computational resources and training: The project will require hardware acceleration (GPUs with a minimum of 16 GB VRAM). The estimated training and fine-tuning time for the base models is approximately 15 hours, while the main process of optimizing perceptual perturbations (XAI attack) for the entire test set is estimated to take an additional 25–30 hours of computation.

3. Planned Program Features

• Classification and Attribution Module: Reading models and generating explanation maps for them.
• Perturbation module: Generating subtle modifications to the audio signal with optimization that preserves high perceptual metrics (PESQ, PEAQ, STOI and ViSQOL).
• Deployment and Automation: Scripted building, testing, and deployment of applications using tools such as just and Python scripts built with typer or argparse.
• Final deliverables: The project will include clear documentation, user instructions, and tests relevant to the project’s scope.

4. Planned Technology Stack

The project will implement a robust base structure, automatically generated by tool cookiecutter.

• Environment management: Use of an isolated virtual environment managed by uv or conda (SLURM-managed cluster will be used).
• Code cleanliness: Enforced PEP8-compliant coding style with an increased line length limit. Syntax checking provided by an autoformatter and a linter using ruff.
• Version control: Rigorous use of a code repository with the conventional commits specification implemented.
• Frameworks and AI: Implementation of learning logic in dedicated frameworks such as PyTorch Lightning in conjunction with Huggingface libraries. Code used for experiments will be continuously exported from Jupyter Lab notebooks into structured library code.
• Experiments and configuration: Tracking progress, metrics, and logs using the Tensorboard platform. The configuration of model parameters and experiments will be completely separated from the execution code.
• Documentation: Use of mkdocs to fast and simple write documentation

5. Project schedule