stream-translator-gpt

Command line utility to transcribe or translate audio from livestreams in real time. Uses yt-dlp to get livestream URLs from various services and Whisper / Faster-Whisper for transcription.

This fork optimized the audio slicing logic based on VAD, introduced GPT API / Gemini API to support language translation beyond English, and supports input from the audio devices.

Try it on Colab:

Prerequisites

Linux or Windows:

Python >= 3.8 (Recommend >= 3.10)
Install CUDA on your system. You can check the installed CUDA version with nvcc --version.
Install cuDNN to your CUDA dir if you want to use Faseter-Whisper.
Install PyTorch (with CUDA) to your Python.
Create a Google API key if you want to use Gemini API for translation. (Recommend, Free 60 requests / minute)
Create a OpenAI API key if you want to use Whisper API for transcription or GPT API for translation.

If you are in Windows, you also need to:

Install and add ffmpeg to your PATH.
Install yt-dlp and add it to your PATH.

Installation

Install release version from PyPI (Recommend):

pip install stream-translator-gpt
stream-translator-gpt

Clone master version code from Github:

git clone https://github.com/ionic-bond/stream-translator-gpt.git
pip install -r ./stream-translator-gpt/requirements.txt
python3 ./stream-translator-gpt/translator.py

Usage

Transcribe live streaming (default use Whisper):

stream-translator-gpt {URL} --model large --language {input_language}
Transcribe by Faster Whisper:

stream-translator-gpt {URL} --model large --language {input_language} --use_faster_whisper
Transcribe by Whisper API:

stream-translator-gpt {URL} --language {input_language} --use_whisper_api --openai_api_key {your_openai_key}
Translate to other language by Gemini:

stream-translator-gpt {URL} --model large --language ja --gpt_translation_prompt "Translate from Japanese to Chinese" --google_api_key {your_google_key}
Translate to other language by GPT:

stream-translator-gpt {URL} --model large --language ja --gpt_translation_prompt "Translate from Japanese to Chinese" --openai_api_key {your_openai_key}
Using Whisper API and Gemini at the same time:

stream-translator-gpt {URL} --model large --language ja --use_whisper_api --openai_api_key {your_openai_key} --gpt_translation_prompt "Translate from Japanese to Chinese" --google_api_key {your_google_key}
Local video/audio file as input:

stream-translator-gpt /path/to/file --model large --language {input_language}
Computer microphone as input:

stream-translator-gpt device --model large --language {input_language}

Will use the system's default audio device as input.

If you want to use another audio input device, stream-translator-gpt device --print_all_devices get device index and then run the CLI with --device_index {index}.

If you want to use the audio output of another program as input, you need to enable stereo mix.
Sending result to Cqhttp:

stream-translator-gpt {URL} --model large --language {input_language} --cqhttp_url {your_cqhttp_url} --cqhttp_token {your_cqhttp_token}
Sending result to Discord:

stream-translator-gpt {URL} --model large --language {input_language} --discord_webhook_url {your_discord_webhook_url}

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

stream-translator-gpt-2024.3.22.tar.gz (1.1 MB view details)

Uploaded Mar 22, 2024 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

stream_translator_gpt-2024.3.22-py3-none-any.whl (1.1 MB view details)

Uploaded Mar 22, 2024 Python 3

File details

Details for the file stream-translator-gpt-2024.3.22.tar.gz.

File metadata

Download URL: stream-translator-gpt-2024.3.22.tar.gz
Upload date: Mar 22, 2024
Size: 1.1 MB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for stream-translator-gpt-2024.3.22.tar.gz
Algorithm	Hash digest
SHA256	`640b7053ec799519687fd5f3bbb9865440f3d1517928c5d9c5789e278c40a92d`
MD5	`e5a61eb997559eb495bdee7db52393fd`
BLAKE2b-256	`ef5d10ceeac5c3643a3f8b71592b41a3ca3dbbab5c5b53b0a3bd5d4def27f592`

See more details on using hashes here.

File details

Details for the file stream_translator_gpt-2024.3.22-py3-none-any.whl.

File metadata

Download URL: stream_translator_gpt-2024.3.22-py3-none-any.whl
Upload date: Mar 22, 2024
Size: 1.1 MB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for stream_translator_gpt-2024.3.22-py3-none-any.whl
Algorithm	Hash digest
SHA256	`0271a4de5c4f5cc152f95c731c9844829fc04a5beebbd367fc642e17256945e7`
MD5	`4cf6531ca27bb35ae180fc55baa1231e`
BLAKE2b-256	`bd75e3a0f3a7daa142c5864b91121aa02974676e8d380d94dfdd180afd4cd19f`