Skip to main content

Differentiable audio processors in PyTorch.

Project description

dasp

Differentiable audio signal processors in PyTorch

  Includes reverberation, distortion, dynamic range processing, equalization, stereo processing.

  Enables virtual analog modeling, blind parameter estimation, automated DSP, and style transfer.

  Batching with operation on both CPU and GPU accelerators for fast training and reduced bottlenecks.

  Open source and free to use for academic and commercial applications under Apache 2.0 license.

Installation

git clone https://github.com/csteinmetz1/dasp-pytorch
cd dasp-pytorch
pip install -e .

Note: Coming to PyPi soon to enable pip install dasp-pytorch.

Examples

dasp-pytorch is a Python library for constructing differentiable audio signal processors using PyTorch. These differentiable processors can be used standalone or within the computation graph of neural networks. We provide purely functional interfaces for all processors that enables ease-of-use and portability across projects. Unless oterhwise stated, all effect functions expect 3-dim tensors with shape (batch_size, num_channels, num_samples) as input and output. Using an effect in your computation graph is as simple as calling the function with the input tensor as argument.

Quickstart

Here is a minimal example to demonstrate reverse engineering the drive value of a simple distortion effect using gradient descent.

import torch
import torchaudio
import dasp_pytorch

# Load audio
x, sr = torchaudio.load("audio/short_riff.wav")

# create batch dim
# (batch_size, n_channels, n_samples)
x = x.unsqueeze(0)

# apply some distortion with 16 dB drive
drive = torch.tensor([16.0])
y = dasp_pytorch.functional.distortion(x, drive)

# create a parameter to optimizer
drive_hat = torch.nn.Parameter(torch.tensor(0.0))
optimizer = torch.optim.Adam([drive_hat], lr=0.01)

# optimize the parameter
n_iters = 2500
for n in range(n_iters):
    # apply distortion with the estimated parameter
    y_hat = dasp_pytorch.functional.distortion(x, drive_hat)

    # compute distance between estimate and target
    loss = torch.nn.functional.mse_loss(y_hat, y)

    # optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print(
        f"step: {n+1}/{n_iters}, loss: {loss.item():.3f}, drive: {drive_hat.item():.3f}"
    )

For the remaining examples we will use the GuitarSet dataset. You can download the data using the following commands:

mkdir data
wget https://zenodo.org/records/3371780/files/audio_mono-mic.zip
unzip audio_mono-mic.zip
rm audio_mono-mic.zip

More examples

Audio Processors

Audio Processor Functional Interface
Gain gain()
Distortion distortion()
Parametric Equalizer parametric_eq()
Dynamic range compressor compressor()
Dynamic range expander expander()
Reverberation noise_shaped_reverberation()
Stereo Widener stereo_widener()
Stereo Panner stereo_panner()
Stereo Bus stereo_bus()

Citations

If you use this library consider citing these papers:

Differnetiable parametric EQ and dynamic range compressor

@article{steinmetz2022style,
  title={Style transfer of audio effects with differentiable signal processing},
  author={Steinmetz, Christian J and Bryan, Nicholas J and Reiss, Joshua D},
  journal={arXiv preprint arXiv:2207.08759},
  year={2022}
}

Differentiable artificial reveberation with frequency-band noise shaping

@inproceedings{steinmetz2021filtered,
  title={Filtered noise shaping for time domain room impulse 
         response estimation from reverberant speech},
  author={Steinmetz, Christian J and Ithapu, Vamsi Krishna and Calamia, Paul},
  booktitle={WASPAA},
  year={2021},
  organization={IEEE}
}

Differnetiable IIR filters

@inproceedings{nercessian2020neural,
  title={Neural parametric equalizer matching using differentiable biquads},
  author={Nercessian, Shahan},
  booktitle={DAFx},
  year={2020}
}
@inproceedings{colonel2022direct,
  title={Direct design of biquad filter cascades with deep learning 
          by sampling random polynomials},
  author={Colonel, Joseph T and Steinmetz, Christian J and 
          Michelen, Marcus and Reiss, Joshua D},
  booktitle={ICASSP},
  year={2022},
  organization={IEEE}

Acknowledgements

Supported by the EPSRC UKRI Centre for Doctoral Training in Artificial Intelligence and Music (EP/S022694/1).

           

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dasp-pytorch-0.0.1.tar.gz (19.8 kB view details)

Uploaded Source

Built Distribution

dasp_pytorch-0.0.1-py3-none-any.whl (18.5 kB view details)

Uploaded Python 3

File details

Details for the file dasp-pytorch-0.0.1.tar.gz.

File metadata

  • Download URL: dasp-pytorch-0.0.1.tar.gz
  • Upload date:
  • Size: 19.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for dasp-pytorch-0.0.1.tar.gz
Algorithm Hash digest
SHA256 23a3853550bcc7caa0964440a83588662e6cb2d0aa458cdfc4aabd1afa1c9eba
MD5 2f08c6829155a7bfdb5a6d41930f0a7b
BLAKE2b-256 915eca9bd836fa78b0d2e9d7561f33e52ff3014db22600bc05719bd68432fe22

See more details on using hashes here.

File details

Details for the file dasp_pytorch-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: dasp_pytorch-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 18.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/4.0.2 CPython/3.11.6

File hashes

Hashes for dasp_pytorch-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 2116a253c40dbab74eecfc93ebae9c00f0bb7df698c2cc1f9796aeaef5cb6f48
MD5 76da0043323b2405a85b7271eca63ee3
BLAKE2b-256 c3b4b3a3f60af0bc82178fce8d99328c1504e4c732e3fb85aece8fddd3943a8c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page