hibiki-zero 0.0.4

Simultaneous Speech-to-Speech Translation Without Aligned Data.

Hibiki-Zero

Hibiki-Zero is a real-time and multilingual speech translation model. It translates from French, Spanish, Portuguese and German to English: accurately, with low latency, high audio quality, and voice transfer.

https://github.com/user-attachments/assets/d533ec45-8d5e-4e41-886a-0b2d198be6f3

🤗 Hugging Face Model Card | ⚙️ Tech report | 📄 Paper | 🎧 More samples

Requirements

Hibiki-Zero is a 3B-parameter model and requires an NVIDIA GPU to run: 8 GB VRAM should work, 12 GB is safe.

Run the server

Hibiki-Zero comes with a server you can run to interact with Hibiki in real time. To run it, just use:

uvx -p 3.13 hibiki-zero serve [--gradio-tunnel]

Then go to the URL displayed to try out Hibiki-Zero. The --gradio-tunnel flag will forward the server to a public URL that you can access from anywhere.

If you don't have uv, you must first install hibiki-zero with pip install hibiki-zero and then run the server with hibiki-zero serve [--gradio-tunnel].

Run inference

If you'd like to run Hibiki-Zero on existing audio files, run:

uvx -p 3.13 hibiki-zero generate [--file /path/to/my/audio.wav --file /path/to/another/audio.mp3]

Batch inference is supported, meaning you can run the model on multiple audio files at the same time.

Local development

We recomment using uv, run anything with uv run in this repository. For example

uv run some_file.py
or 
uv run hibiki-zero serve

if you use pip, use pip install -e . before executing python commands.