A high-quality general neural audio codec.
Project description
Descript Audio Codec (.dac): High-Fidelity Audio Compression with Improved RVQGAN
This repository contains training and inference scripts for the Descript Audio Codec (.dac), a high fidelity general neural audio codec, introduced in the paper titled High-Fidelity Audio Compression with Improved RVQGAN.
arXiv Paper: High-Fidelity Audio Compression with Improved RVQGAN
📈 Demo Site
⚙ Model Weights
👉 With Descript Audio Codec, you can compress 44.1 KHz audio into discrete codes at a low 8 kbps bitrate.
🤌 That's approximately 90x compression while maintaining exceptional fidelity and minimizing artifacts.
💪 Our universal model works on all domains (speech, environment, music, etc.), making it widely applicable to generative modeling of all audio.
👌 It can be used as a drop-in replacement for EnCodec for all audio language modeling applications (such as AudioLMs, MusicLMs, MusicGen, etc.)
Usage
Installation
pip install descript-audio-codec
OR
pip install git+https://github.com/descriptinc/descript-audio-codec
Weights
Weights are released as part of this repo under MIT license.
They are automatically downloaded when you first run encode
or decode
command. They can be cached locally with
python3 -m dac download
We provide a Dockerfile that installs all required dependencies for encoding and decoding. The build process caches model weights inside the image. This allows the image to be used without an internet connection. Please refer to instructions below.
Compress audio
python3 -m dac encode /path/to/input --output /path/to/output/codes
This command will create .dac
files with the same name as the input files.
It will also preserve the directory structure relative to input root and
re-create it in the output directory. Please use python -m dac encode --help
for more options.
Reconstruct audio from compressed codes
python3 -m dac decode /path/to/output/codes --output /path/to/reconstructed_input
This command will create .wav
files with the same name as the input files.
It will also preserve the directory structure relative to input root and
re-create it in the output directory. Please use python -m dac decode --help
for more options.
Programmatic Usage
import dac
from dac.utils import load_model
from dac.model import DAC
from dac.utils.encode import process as encode
from dac.utils.decode import process as decode
from audiotools import AudioSignal
# Init an empty model
model = DAC()
# Load compatible pre-trained model
model = load_model(dac.__model_version__)
model.eval()
model.to('cuda')
# Load audio signal file
signal = AudioSignal('input.wav')
# Encode audio signal
encoded_out = encode(signal, 'cuda', model)
# Decode audio signal
recon = decode(encoded_out, 'cuda', model, preserve_sample_rate=True)
# Write to file
recon.write('recon.wav')
Docker image
We provide a dockerfile to build a docker image with all the necessary dependencies.
-
Building the image.
docker build -t dac .
-
Using the image.
Usage on CPU:
docker run dac <command>
Usage on GPU:
docker run --gpus=all dac <command>
<command>
can be one of the compression and reconstruction commands listed above. For example, if you want to run compression,docker run --gpus=all dac python3 -m dac encode ...
Training
The baseline model configuration can be trained using the following commands.
Pre-requisites
Please install the correct dependencies
pip install -e ".[dev]"
Single GPU training
export CUDA_VISIBLE_DEVICES=0
python scripts/train.py --args.load conf/ablations/baseline.yml --save_path runs/baseline/
Multi GPU training
export CUDA_VISIBLE_DEVICES=0,1
torchrun --nproc_per_node gpu scripts/train.py --args.load conf/ablations/baseline.yml --save_path runs/baseline/
Testing
We provide two test scripts to test CLI + training functionality. Please make sure that the trainig pre-requisites are satisfied before launching these tests. To launch these tests please run
python -m pytest tests
Results
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for descript-audio-codec-0.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2c1bb6631ec3219f22220505ac2101514442a5615d63c7021b258d46f15da1ce |
|
MD5 | e5b7d6f91207ad2b26537c38b2c3cae3 |
|
BLAKE2b-256 | dd77a7a3b3a7cb9aad8f8d4565f9b0b50a1dcb839a7be67e7ca8075a4966000d |
Hashes for descript_audio_codec-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b1acec1e7742e3ff16332c9bf94eceeabe361b7caf7b0229316510674b4f1d83 |
|
MD5 | e082acb2e19ef9f8c93b4fe31c69ca3a |
|
BLAKE2b-256 | 8d9ee677a5fcb60163537aded868c6297d5dfa571b8c944b852800927c3a5d93 |