lyriks-video 0.2.1

Automated lyrics video generator

Lyriks

Lyriks is an automated lyrics video generator. It transcribes the audio and automatically creates a video using MoviePy.

---

Features

• Automatic vocal separation using Demucs
• Transcription with OpenAI Whisper and whisper-timestamped
• Synchronized lyrics video generation with MoviePy

---

Requirements

• A NVIDIA GPU
• 10GB of free disk space

---

Installation

1. Clone the repository:

   git clone https://github.com/simon0302010/Lyriks.git
   cd Lyriks
   

2. (Optional) Create a virtual environment:

   python3 -m venv venv
   source venv/bin/activate
   

3. Install Lyriks:

   pip install .
   

---

Usage

python -m lyriks generate path/to/song.mp3 path/to/lyrics.json -m WHISPER_MODEL_SIZE -d DEVICE -o OUTPUT_FILE_NAME

Note: This process can take up to 20 minutes on lower end hardware.

---

TODO

• Fix up lyrics using Gemini
• Per-word highlighting in videos
• Fancier video styles and effects
• Add more robust error handling

---

Credits

This project uses Demucs for music vocal separation.

@inproceedings{rouard2022hybrid,
  title={Hybrid Transformers for Music Source Separation},
  author={Rouard, Simon and Massa, Francisco and D{'e}fossez, Alexandre},
  booktitle={ICASSP 23},
  year={2023}
}

@inproceedings{defossez2021hybrid,
  title={Hybrid Spectrogram and Waveform Source Separation},
  author={D{'e}fossez, Alexandre},
  booktitle={Proceedings of the ISMIR 2021 Workshop on Music Source Separation},
  year={2021}
}