lyriks-video 0.5.0

Automated lyrics video generator

Lyriks

Lyriks is an automated lyrics video generator. It transcribes audio and automatically creates a lyrics video using fast subtitle rendering (pysubs2+ffmpeg) or MoviePy.

---

Features

• Automatic vocal separation using Demucs
• Transcription with OpenAI Whisper and whisper-timestamped
• Fast, high-quality video rendering with pysubs2 + FFmpeg
• Synchronized lyrics video generation with MoviePy (legacy)
• ASS subtitle generation with pysubs2
• Fast video rendering using FFmpeg

---

Requirements

• Linux (Windows support is experimental; macOS hasn't been tested yet)
• An NVIDIA GPU (recommended for best performance; CPU is supported but slower)
• 10GB of free disk space
• Python 3.11
• ffmpeg

Installing FFmpeg

On Ubuntu/Debian:

sudo apt update
sudo apt install ffmpeg

On Arch Linux:

sudo pacman -S ffmpeg

For other platforms and more details, see the FFmpeg download page.

---

Installation

It is highly recommended to use a virtual environment for isolation:

python3 -m venv .venv
source .venv/bin/activate

Then install Lyriks with pip:

pip install lyriks-video

---

Usage

python -m lyriks generate AUDIO_FILE LYRICS_FILE [OPTIONS]

Parameters

• AUDIO_FILE
  Path to the input audio file (e.g., song.mp3).
  This should be a supported audio format (such as MP3 or WAV).

• LYRICS_FILE
  Path to the lyrics file (plain text).
  The lyrics should be in a text file, one line per lyric segment.

Options

You will be interactively prompted in the CLI for any options you leave unspecified.

• --output, -o
  Output video file name (without extension).
  Example: -o my_lyrics_video

• --model_size, -m
  Sets the Whisper model size for transcription.
  Options: tiny, base, small, medium, large, turbo

• --device, -d
  Which device to use for Whisper model inference.
  Options: cpu, cuda

• --generator, -g
  Which backend to use for video generation.
  Options:

  • ps2: pysubs2 + ffmpeg (fast, good quality, experimental, ~60 fps)
  • mp: MoviePy (slow, low quality, legacy, ~10 fps)
  • ts: Only save transcript (for debugging)

• --background, -b
  Optional background video file for the video (must be a video the same length or longer than the audio).
  Example: -b my_background.mp4

• --no-gemini
  Disable Gemini improvements for Whisper output.

---

Example

python -m lyriks generate path/to/song.mp3 path/to/lyrics.txt -m small -d cuda -o output_video -b background.mp4

Note: This process can take up to 5 minutes on lower end hardware.

---

TODO

• Fancier video styles and effects
• Add more robust error handling
• Ask which background to use
• Libary of procedually generated backgrounds
• Batch processing
• Loading bars
• Karaoke function
• Automatic upload to YouTube

---

How Lyriks Works

---

Credits

This project uses:

• Demucs for music vocal separation.
• whisper-timestamped for word-level timestamped transcription.

---

Citations

If you use this in your research, please cite the following:

Demucs

@inproceedings{rouard2022hybrid,
  title={Hybrid Transformers for Music Source Separation},
  author={Rouard, Simon and Massa, Francisco and D{'e}fossez, Alexandre},
  booktitle={ICASSP 23},
  year={2023}
}

@inproceedings{defossez2021hybrid,
  title={Hybrid Spectrogram and Waveform Source Separation},
  author={D{'e}fossez, Alexandre},
  booktitle={Proceedings of the ISMIR 2021 Workshop on Music Source Separation},
  year={2021}
}

whisper-timestamped

@misc{lintoai2023whispertimestamped,
  title={whisper-timestamped},
  author={Louradour, J{\'e}r{\^o}me},
  journal={GitHub repository},
  year={2023},
  publisher={GitHub},
  howpublished = {\url{https://github.com/linto-ai/whisper-timestamped}}
}

OpenAI Whisper

@article{radford2022robust,
  title={Robust speech recognition via large-scale weak supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal={arXiv preprint arXiv:2212.04356},
  year={2022}
}

Dynamic-Time-Warping

@article{JSSv031i07,
  title={Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package},
  author={Giorgino, Toni},
  journal={Journal of Statistical Software},
  year={2009},
  volume={31},
  number={7},
  doi={10.18637/jss.v031.i07}
}

---

License

This project is licensed under the GPL-3.0 License.

---

Contributing

Contributions are welcome!
If you have suggestions, bug reports, or want to add features, please open an issue or submit a pull request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/my-feature)
3. Commit your changes (git commit -am 'Add new feature')
4. Push to the branch (git push origin feature/my-feature)
5. Open a pull request

---

Contact

For questions, bug reports, or feedback, please open an issue on GitHub
or contact the maintainer: simon0302010 (GitHub username).

---