X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning

X-Voice is a flow-matching-based multilingual zero-shot voice cloning system that enables one speaker to speak 30 languages.

News

• 2026/04/30: X-Voice codebase, model, demo, dataset, and benchmark are released.

Installation

Create a separate environment if needed

# Create a conda env with python_version>=3.11
conda create -n x-voice python=3.11
conda activate x-voice

# Install FFmpeg if you haven't yet
conda install ffmpeg

Install PyTorch with matched device

Install X-Voice

git clone https://github.com/sunnyxrxrx/X-Voice.git
cd X-Voice
pip install -e .

Check your ESpeak-ng installation:

espeak-ng --version

If not found, run src/x_voice/prepare_ipa.sh first.

Inference

• In order to achieve desired performance, take a moment to read detailed guidance.

1. Gradio App

x-voice_infer-gradio --host 0.0.0.0 --port 7860

2. CLI Inference

# X-Voice Stage1
python -m x_voice.infer.infer_cli_stage1 -c src/x_voice/infer/examples/basic/basic_stage1.toml

# X-Voice Stage2
python -m x_voice.infer.infer_cli_stage2 -c src/x_voice/infer/examples/basic/basic_stage2.toml

Training

TTS Model Training

Refer to training guidance for best practice.

Speaking Rate Predictor Training

Refer to speaking rate predictor guidance for the multilingual speaking rate predictor used in X-Voice.

Evaluation

Refer to evaluation guidance for benchmark and metric scripts.

Repo Structure

X-Voice/
├── ckpts/                  # checkpoints
├── data/                   # datasets and processed data
├── src/
│   ├── rate_pred/          # speaking rate predictor
│   ├── third_party/
│   │   └── BigVGAN/        # BigVGAN submodule
│   └── x_voice/            # main X-Voice package
└── pyproject.toml          # package definition and dependencies

Development

Use pre-commit to ensure code quality:

pip install pre-commit
pre-commit install
pre-commit run --all-files

Acknowledgements

• F5-TTS brilliant work and the foundation of this codebase
• Cross-Lingual F5-TTS 2 for its supervised fine-tuning strategy with synthetic audio prompts
• Cross-Lingual F5-TTS for its speaking rate predictor
• NLLB for translation in the Gradio demo
• torchdiffeq as ODE solver, Vocos and BigVGAN as vocoder
• FunASR, faster-whisper, UniSpeech, SpeechMOS for evaluation tools
• MAVL for Japanese syllable counting

License

Our code is released under MIT License. The pre-trained models are licensed under the CC-BY-NC license due to the training data Emilia, which is an in-the-wild dataset. Sorry for any inconvenience this may cause.