Skip to main content

DeePAQ

DeePAQ uses pre-trained MERT to extract embeddings of size 256. The provided training_mapping.py maps embeddings to a MUSHRA-like scale (0–100).

Table of Contents

Installation

DeePAQ is built and tested on PyTorch 2.5.1, TorchVision 0.20.1, TorchAudio 2.5.1 and CUDA 12.1 runtime It can be either installed with pip or git.

pip

Checkpoints are not part of the PyPI package. Download the DeePAQ checkpoints, and the neural mapping checkpoint if you use mapping: nn, and point model_path / mapping_model_path at those files.

pip install DeePAQ

From a local build of this repository:

pip install dist/deepaq-0.1.0-py3-none-any.whl

git

git clone https://git.audiolabs.uni-erlangen.de/guanxin.jiang/audio-quality-assessment-dnn.git
cd audio-quality-assessment-dnn
git checkout DeePAQ
conda env create -f environment.yml
conda activate DeePAQ-env

Dataset

For evaluation the project requires signals under test, matched reference signals, and corresponding subjective scores in CSV format. Each CSV should include columns:

  • filepath_deg — path to the degraded/test signal
  • filepath_ref — (optional) path to the matched reference signal
  • SG — (optional) subjective rating. If SG column exists in your csv file, then the script will automatically detect it and then calculate the overall Spearman and Pearson correlation between the mapped results and subjective ratings

Place your dataset files in the data/ folder.

Usage

Training

Run the main script with your training config:

python main.py --config_file src/config/train1_1.yaml

Evaluation

Extract Embeddings

If you pip install deepaq

import deepaq
wav_path='/path/test.wav'
checkpoint='/path/best_model.pt'
embs = deepaq.compute_embedding(wav_path, checkpoint, model_type='MERT', mapping_type='polynomial')

Full Reference

python main.py --config_file src/config/fr_eval.yaml

or if you pip install deepaq

deepaq --config_file src/config/fr_eval.yaml

In fr_eval.yaml config file

  • model — foundation model
  • model_path — checkpoint has been provided in out-models folder. For full reference, 2 checkpoints are provided, which are MERT-FR-2f and MERT-FR-best
  • mapping - choose the mapping functions: polynomial uses 3rd order polynomial mapping; spline uses quadratic B-spline (degree 2, order 3) with three internal knots; nn uses a neural network trained to map Euclidean distance to MUSHRA scale.
  • mapping_model_path: ./out-models/Dist2SG_Mapping/best_model.pt (This is optional if polynomial or spline is used for mapping)
  • test_files_csv — csv file containing the data for testing
  • test_root_wav — folder where your test data stored
  • emb_out_dir - folder where you want to save your evalution results
  • emb_out_file — csv file saves the evaluation results
  • plot_out_file — png file saves scatterplot of the evaluation results

Non-matching Reference

python main.py --config_file src/config/nmr_eval.yaml

In nmr_eval.yaml config file

  • model — foundation model
  • model_path — checkpoint has been provided in out-models folder.
  • mapping - choose the mapping functions: polynomial uses 3rd order polynomial mapping; spline uses quadratic B-spline (degree 2, order 3) with three internal knots; nn uses a neural network trained to map Euclidean distance to MUSHRA scale.
  • mapping_model_path: ./out-models/Dist2SG_Mapping/best_model.pt (This is optional if polynomial or spline is used for mapping)
  • test_files_csv — csv file containing the data for testing
  • test_root_wav — folder where your test data stored
  • emb_out_dir - folder where you want to save your evalution results
  • emb_out_file — csv file saves the evaluation results
  • plot_out_file — png file saves scatterplot of the evaluation results
  • non_matching_Mref_root — folder where contains the non-matching reference with music content
  • non_matching_Sref_root — folder where contains the non-matching reference with speech content

Notation

model wav2vec is for development. If users want to use DeePAQ for inference, wav2vec could be skipped; otherwise need to install fairseq for wav2vec

Metadata

Release files for DeePAQ 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for DeePAQ 0.1.0
File Size Uploaded Provenance Actions
deepaq-0.1.0.tar.gz 11.9 kB Without provenance Details for deepaq-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for DeePAQ 0.1.0
File Interpreter ABI Platform Actions
deepaq-0.1.0-py3-none-any.whl Python 3 none any Details for deepaq-0.1.0-py3-none-any.whl

Total release size: 25.2 kB

Release files / deepaq-0.1.0.tar.gz

Download URL deepaq-0.1.0.tar.gz
Size 11.9 kB
Tags Source
SHA-256 checksum
How to use checksums
7d6e4ae2c178d685e0f3ad232befd5dcf4d8ea0a0cc782112e7a8b5c874df637
BLAKE2b-256 checksum
How to use checksums
bb0a77d3ed2f3541115768271094bb78dc07245c8326b83c4a76000b000e3399
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.19

Release files / deepaq-0.1.0-py3-none-any.whl

Download URL deepaq-0.1.0-py3-none-any.whl
Size 13.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e8b7329eef7a72bf946c02265a63ed4ef35e3f60ba3169e9959ace47d71a3bdd
BLAKE2b-256 checksum
How to use checksums
a70887e4fb4fea5044a920470bb4c2e7df0719b4c27b00a20bd8d5b86d5853d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.19

Security for DeePAQ 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page