A vocoder that can convert audio to Mel-Spectrogram and reverse with WaveGlow, all on GPU(if avaliable).
Project description
WaveGlow Vocoder
A vocoder that can convert audio to Mel-Spectrogram and reverse with WaveGlow, all on GPU.
Most code is from Tacotron2 and WaveGlow.
Install
pip install waveglow-vocoder
Example
Original | Vocoded |
---|---|
original music | vocoded music |
original speech | vocoded speech |
Usage
Load wav file as torch tensor on GPU.
import librosa
y,sr = librosa.load(librosa.util.example_audio_file(), sr=22050, mono=True, duration=10, offset=30)
y_tensor = torch.from_numpy(y).to(device='cuda', dtype=torch.float32)
Apply mel transform, this would be done on GPU(if ava).
from waveglow_vocoder import WaveGlowVocoder
WV = WaveGlowVocoder()
mel = WV.wav2mel(y_tensor)
Decoder it with Waveglow.
NOTE:
As the hyperparameter of pre-trained model is alignment with Tacotron2, one might get totally noise if the Mel spectrogram comes from other function than wav2mel(an alias for TacotronSTFT.mel_spectrogram).
Support for the melspectrogram from librosa and torchaudio is under development.
wav = WV.mel2wav(mel)
Other pretrained model / Train with your own data
This vocoder will download pre-trained model from pytorch hub on the first time of initialize.
You can also download the latest model from WaveGlow, or with your own data and pass the path to the waveglow vocoder.
config_path = "your_config_of_model_training.json"
waveglow_path="your_model_path.pt"
WV = WaveGlowVocoder(waveglow_path=waveglow_path, config_path=config_path)
Then use it as usual.
TODO
- WaveRNN Vocoder
- MelGAN Vocoder
- Performance
- Support librosa Mel input
Reference
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.