TTS-with-RVC-ONNX 0.1.10

TTS-with-RVC-ONNX (Text-to-Speech with RVC using ONNX) is a package designed to enhance the capabilities of text-to-speech (TTS) systems by introducing an RVC module running on the ONNX Runtime. The package enables users to not only convert text into speech but also personalize and customize the voice output according to their preferences with RVC support, optimized for various hardware backends (DirectML, CUDA, CPU).

ONNX Runtime is used for RVC inference, potentially leveraging hardware acceleration (DirectML on Windows/AMD, CUDA on NVIDIA). PyTorch is also required for audio processing; RMVPE itself uses its ONNX model.

It may contain bugs. Report an issue in case of error.

Release notes

0.1.10 - September 30, 2026: Added MultiGPU support: run independent instances on different GPUs in the same process. Explicit CUDA/DirectML adapter selection, separate ONNX sessions and predictor caches per instance, async usage with async_call(), safe device switching, and resource cleanup with close() / context managers.

0.1.9 - April 10, 2025: Current ONNX Branch Sync

• Synced RVC parameters with main branch 0.1.9 (rms_mix_rate, protect, filter_radius, resample_sr, file_index2, verbose).
• Added support for F0 predictors: rmvpe (using ONNX), pm, dio, harvest.
• Fixed F0 length mismatch issue and implemented correct audio padding.
• Added set_device method to switch ONNX Runtime providers.
• Updated dependencies and ONNX Runtime selection.

(Based on main branch 0.1.9)

0.1.6 - Initial ONNX support.

Prerequisites

You must have Python 3.10, 3.11 or 3.12 installed. You must have ONNX Runtime compatible hardware/drivers if using GPU acceleration (DirectML for AMD on Windows, CUDA for NVIDIA). The CPU provider works generally.

• PyTorch is a package dependency used for audio processing.
• FFmpeg must be installed and accessible in your system's PATH or placed in the script's directory. Download from ffmpeg.org.

Installation

1. Install the package using pip: CPU Version:

   pip install tts-with-rvc-onnx
   

   CUDA version:

   pip install tts-with-rvc-onnx[cuda]
   

   DML version (recommended for AMD):

   pip install tts-with-rvc-onnx[dml]
   

2. Ensure FFmpeg is installed and accessible (see Prerequisites).

How it Works

1. Text-to-Speech (TTS): Uses edge-tts to convert input text into speech, saved as a temporary audio file in the tmp_directory.
2. RVC (ONNX): With the .onnx file provided, the RVC module (via ONNX Runtime) reads the temporary audio file, processes it (feature extraction, F0, conversion, index lookup), and generates a new audio file saved in output_directory with the voice replaced.

Usage

TTS-with-RVC-ONNX has a class called TTS_RVC.

Constructor Parameters:

• model_path (str): Required. Path to your .onnx RVC model file.

And optional parameters:

• voice (str): Voice from edge-tts list (default: "ru-RU-DmitryNeural").

• device (str): ONNX Runtime provider ("dml", "cuda:0", "cpu", etc.). Defaults to "dml".

• tmp_directory (str): Path to directory for temporary TTS files (default: tts_with_rvc_onnx in the system temp folder).

• output_directory (str): Directory for saving final voiced audio (default: tts_with_rvc_onnx/output in the system temp folder).

• index_path (str): Path to the Faiss .index file for voice adjustments (default: "").

• f0_method (str): Method for calculating pitch. Available: 'rmvpe', 'pm', 'harvest', 'dio'. Defaults to "pm".

• sampling_rate (int): Target sample rate of the RVC model (default: 40000).

• hop_size (int): Hop size of the RVC model (default: 512).

Deprecated:

• input_directory: Use tmp_directory instead.

Initialization Example:

from tts_with_rvc import TTS_RVC

tts = TTS_RVC(model_path="models/YourModel.onnx",
                index_path="logs/YourIndex.index",
                f0_method="rmvpe",
                device="dml") # Or "cuda:0", "cpu"

Both the ONNX and PyTorch distributions use the tts_with_rvc import name. Install them in separate environments.

Next, set the voice for TTS with tts.set_voice() function:

tts.set_voice("ru-RU-DmitryNeural")

Setting the appropriate language is necessary if you are using other languages for voiceovers!

And final step is calling tts (the __call__ method) to generate and replace voice:

path = tts(text="Привет, мир!", pitch=6, index_rate=0.50)

__call__ Parameters:

• text (str): Required. Text for TTS.

• pitch (int, optional): Pitch change (transpose) for RVC in semitones. Negative values compatible. Default: 0.

• tts_rate (int, optional): Extra rate of speech for Edge TTS in percentage (+/-). Default: 0.

• tts_volume (int, optional): Extra volume of speech for Edge TTS in percentage (+/-). Default: 0.

• tts_pitch (int, optional): Extra pitch of TTS-generated audio in Hz (+/-). Not recommended. Default: 0.

• output_filename (str, optional): Name for the output file. If None, a unique name is generated. Default: None.

• index_rate (float, optional): Blending rate between original and indexed voice conversion (0 to 1). Default: 0.75.

• f0method (str, optional): F0 extraction method for this specific call, overrides the instance default: 'rmvpe', 'pm', 'harvest', 'dio'. Default uses instance setting.

• file_index2 (str, optional): Path to secondary index file for RVC. Default: "".

• filter_radius (int, optional): Median filter radius for pitch results. Values >=3 reduce breathiness. Default: 3.

• resample_sr (int, optional): Sample rate to resample final audio to. 0 means use model's sample rate. Default: 0.

• rms_mix_rate (float, optional): Volume envelope scaling (0-1). Lower values mimic original volume more closely. Default: 0.5.

• protect (float, optional): Protection for voiceless consonants and breaths (0-0.5). Lower values increase protection. 0.5 disables. Default: 0.33.

• verbose (bool, optional): Enable verbose logging for RVC conversion. Default: False.

(Note: is_half parameter is removed as precision is handled by ONNX Runtime.)

Example of usage

A simple example for voicing text:

import os
from tts_with_rvc import TTS_RVC
# from playsound import playsound # Optional

# --- Configuration ---
model_file = "models/DenVot.onnx"
index_file = "logs/added_IVF1749_Flat_nprobe_1.index" # Optional
temp_dir = "audio_temp"
output_dir = "audio_output"

os.makedirs(temp_dir, exist_ok=True)
os.makedirs(output_dir, exist_ok=True)

# --- Initialize ---
try:
    tts = TTS_RVC(
        model_path=model_file,
        index_path=index_file,
        tmp_directory=temp_dir,
        output_directory=output_dir,
        device="dml", # Or 'cuda:0', 'cpu'
        f0_method="rmvpe"
    )

    tts.set_voice("ru-RU-DmitryNeural")

    # --- Generate ---
    path = tts(text="Привет, мир!", pitch=6, index_rate=0.9)
    print(f"Audio saved to: {path}")

    # --- Play (Optional) ---
    # playsound(path)

except Exception as e:
    print(f"An error occurred: {e}")

New usage (0.1.10)

MultiGPU

Create one instance per GPU. Each instance owns its model state; distribute requests between them in your application.

from concurrent.futures import ThreadPoolExecutor
from tts_with_rvc import TTS_RVC

with TTS_RVC(
    model_path="models/voice.onnx", device="cuda:0", output_directory="output/gpu0",
) as first, TTS_RVC(
    model_path="models/voice.onnx", device="cuda:1", output_directory="output/gpu1",
) as second:
    with ThreadPoolExecutor(max_workers=2) as pool:
        jobs = [
            pool.submit(first, "First request."),
            pool.submit(second, "Second request."),
        ]
        output_paths = [job.result() for job in jobs]
        print(output_paths)

For DirectML, use dml:0 and dml:1 with a DirectML-enabled ONNX Runtime. A bare cuda selects GPU zero. Use separate instances for parallel conversion; each converter serializes its own conversion calls.

Async applications

import asyncio
from tts_with_rvc import TTS_RVC

async def main():
    with TTS_RVC(model_path="models/voice.onnx", device="cpu") as tts:
        path = await tts.async_call("Hello, world!")
        print(path)

asyncio.run(main())

Changing settings and releasing resources

tts.set_model(path) changes the RVC voice model. Use tts.set_device("cuda:1") to change the device. You can direct auxiliary model downloads to models_dir in the constructor. A with block closes the instance automatically; otherwise call tts.close() when finished.

Text parameters

There are some text parameters processor for integration issues such as adding GPT module.

You can process them using process_args in TTS_RVC class:

• --tts-rate (value): TTS parameter to edit the speech rate.

• --tts-volume (value): TTS parameter to edit the speech volume. May have limited effect due to RVC volume normalization.

• --tts-pitch (value): TTS parameter to edit the pitch of TTS generated audio. Not recommended.

• --rvc-pitch (value): RVC parameter to edit the pitch of the output audio (semitones).

Now the principle of work:

from tts_with_rvc import TTS_RVC

tts = TTS_RVC(model_path="models/YourModel.onnx", device="dml", tmp_directory="temp/")

message_with_args = "This is a test --rvc-pitch -2 and slower --tts-rate -10"

# This method returns arguments and original text without these text parameters
args, clean_message = tts.process_args(message_with_args)
# args = [-10, 0, 0, -2] # [tts_rate, tts_volume, tts_pitch, rvc_pitch]
# clean_message = "This is a test and slower"

# Use extracted arguments for generation:
path = tts(clean_message, tts_rate=args[0],
                        tts_volume=args[1],
                        tts_pitch=args[2],
                        pitch=args[3])

The args variable contains a list with the following structure:

args[0] - TTS Rate

args[1] - TTS Volume

args[2] - TTS Pitch

args[3] - RVC pitch

Methods

• set_voice(voice): Changes the Edge TTS voice.

• set_index_path(index_path): Updates the path to the Faiss .index file.

• set_device(device): Changes the ONNX Runtime provider (e.g., 'dml', 'cuda:0', 'cpu') and reinitializes the backend.

• set_output_directory(directory_path): Sets the default directory for saving output files.

• process_args(text): Extracts text parameters (see above).

• voiceover_file(input_path, ...): Applies RVC voice conversion directly to an existing audio file (accepts same RVC parameters as __call__).

Use tts.set_sampling_params(sr, hop) when changing model sampling settings. The constructor also accepts vec_path and random_seed.

Exceptions

• RuntimeError: Failed to load ONNX model...: Check .onnx model path and integrity. Ensure correct onnxruntime-* package is installed.
• RuntimeError: Failed to initialize ONNX backend...: Check ONNX Runtime installation, drivers (CUDA/DirectML), or model compatibility.
• FileNotFoundError: Input audio, .onnx model, .index file, or required predictor models (rmvpe.onnx) not found.
• ValueError: Dimension mismatch...: Faiss .index file dimension doesn't match ContentVec output dimension (e.g., 256 vs 768). Use a compatible index.
• RuntimeError: Failed to load audio...: Ensure FFmpeg is installed and accessible in PATH.
• Errors during F0 computation: Check if required libraries (parselmouth, pyworld, and the ONNX Runtime provider for RMVPE) are installed correctly.

Acknowledgements

• RVC Project - For the original RVC model and concepts.

License

MIT License

Authors

• Atm4x (Artem Dikarev)