Your one-stop solution for voice dataset creation
Project description
An End-to-End Toolkit for Voice Datasets
VocalForge
is an open-source toolkit written in Python 🐍 that is meant to cut down the time to create datasets for, TTS models, hotword detection models, and more so you can spend more time training, and less time sifting through audio data.
Using Nvidia's NEMO, PyAnnote, CTC segmentation , OpenAI's Whisper, this repo will take you from raw audio to a fully formatted dataset, refining both the audio and text automatically.
NOTE: While this does reduce time on spent on dataset curation, verifying the output at each step is important as it isn't perfect
this is a very much an experimental release, so bugs and updates will be frequent
Features:
audio_demo.ipynb
- ⬇️ Download audio from a YouTube playlist (perfect for podcasts/interviews) OR input your own raw audio files (wav format)
- 🎵 Remove Non Speech Data
- 🗣🗣 Remove Overlapping Speech
- 👥 Split Audio File Into Speakers
- 👤 Isolate the same speaker across multiple files (voice verification)
- 🧽 Use DeepFilterNet to reduce background noise
- 🧮 Normalize Audio
- ➡️ Export with user defined parameters
text_demo.ipynb
- 📜 Batch transcribe text using OpenAI's Whisper
- 🧮 Run text normalization
- 🫶 Use CTC segmentation to line up text to audio
- 🖖 Split audio based on quality of CTC segmentation confidence
- ✅ Generate a metadata.csv and dataset in the format of LJSpeech
Setup/Requirements
Python 3.8 has been tested, newer versions should work
CUDA is required to run all models
a Hugging Face account is required (it's free and super helpful!)
#install system libraries
apt-get update && apt-get install -y libsndfile1 ffmpeg
conda create -n VocalForge python=3.8 pytorch=1.11.0 torchvision=0.12.0 torchaudio=0.11.0 cudatoolkit=11.3.1 -c pytorch
conda activate VocalForge
#to install from pip
pip install VocalForge
#to install source
git clone https://github.com/rioharper/VocalForge
cd VocalForge
pip install -r requirements.txt
#enter huggingface token, token can be found at https://huggingface.co/settings/tokens
huggingface-cli login
Pyannote models need to be "signed up for" in Hugging Face for research purposes. Don't worry, all it asks for is your purpose, website and organization. The following models will have to be manually visited and given the appropriate info:
API Example
from VocalForge.audio import RefineAudio
refine = RefineAudio(
input_dir='raw_audio',
vad_dir='vad',
vad_theshold=0.9
)
refine.VoiceDetection.run()
TODO
- Refactor functions for API and toolkit support
- "Sync" datasets with the metadata file if audio clips are deleted after being generated
- Add a step in the audio refinement processs to remove emotional speech (in progresss)
- Create a model to remove non-speech utterences and portions with background music (in progresss)
- Update code documentation
- Add other normalization methods for audio
- Add other dataset formats for generation
- Utilize TTS models to automatically generate datasets, with audio augmentation to create diversity
- Create a Google Colab Notebook
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for VocalForge-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5b2c56571d6b8df7a4bbab40989391e0c6bcc58eccf9ea7ec42187139bbcc985 |
|
MD5 | b542fae41769439cf532cbf75fa5b761 |
|
BLAKE2b-256 | 6ee7ef1aa19d4695d6e61e8ab3094e67493450cd9ac2deb8d4f8a3e7e9daa2a3 |