Audio ML Spec Tools

Convenience functions for generating ML features from audio data. Breaks audio ML dependencies on torchaudio. Unlike pytorch features, these functions can be exported to ExecuTorch and ONNX with no issues.

Motivation

Except in specific circumstances like wav2vec, raw audio has proven to be a much worse input for ML models than spectrogram-based features across a wide variety of problem domains, including environmental sound classificarion (Guzhov et al. (2021)), singing technique classification (Yamamoto et al. (2021)), and ship classification (Xie, Ren, and Xu (2024)).

There is no scientific consensus on the relative benefits of mel-scale spectrograms, linear spectrograms, and MFCCs. Different researchers have shown good results with each type of spectrogram; see respectively Raponi, Oligeri, and Ali (2021), Jung at al. (2021), and Razani et al (2017).

With this library, you can easily try as many feature extraction methods as you want to see what works for your use case.

Prerequisites

• Python 3.12 runtime
• pip for package installation
• Note that torchcodec depends on a system installation of FFmpeg

Installation

• Install using pip:

pip install AudioMlSpecTools

Local Installation

Install the dependencies and library with pip:

pip install .

Usage

See examples/features.py.

Testing

# If needed, install test dependencies
# pip install .[test]

python3 -m coverage run -m unittest discover -s test -p "*_test.py" && python -m coverage report --skip-covered
python -m coverage html

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Authors

• Ryan Quinn - Initial work

License

MIT.