sinapsis-speech 0.4.6

Generate speech using various libraries.

Sinapsis Speech

A monorepo housing multiple packages and templates for versatile voice generation, text-to-speech, speech-to-text, and beyond.

🐍 Installation • 📦 Packages • 🌐 Webapps • 📙 Documentation • 🔍 License

🐍 Installation

[!IMPORTANT] Sinapsis projects requires Python 3.10 or higher.

This repo includes packages for performing speech synthesis using different tools:

• sinapsis-elevenlabs
• sinapsis-f5-tts
• sinapsis-kokoro
• sinapsis-zonos
• sinapsis-orpheus-cpp
• sinapsis-parakeet

Install using your preferred package manager. We strongly recommend using uv. To install uv, refer to the official documentation.

Install with uv:

uv pip install sinapsis-elevenlabs --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

pip install sinapsis-elevenlabs --extra-index-url https://pypi.sinapsis.tech

Replace sinapsis-elevenlabs with the name of the package you intend to install.

[!IMPORTANT] Templates in each package may require additional dependencies. For development, we recommend installing the package all optional dependencies:

With uv:

uv pip install sinapsis-elevenlabs[all] --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

pip install sinapsis-elevenlabs[all] --extra-index-url https://pypi.sinapsis.tech

Be sure to substitute sinapsis-elevenlabs with the appropriate package name.

[!TIP] You can also install all the packages within this project:

uv pip install sinapsis-speech[all] --extra-index-url https://pypi.sinapsis.tech

📦 Packages

This repository is organized into modular packages, each designed for integration with different text-to-speech tools. These packages provide ready-to-use templates for speech synthesis. Below is an overview of the available packages:

Sinapsis ElevenLabs

This package offers a suite of templates and utilities designed for effortless integrating, configuration, and execution of text-to-speech (TTS), speech-to-speech (STS), voice cloning, and voice generation functionalities powered by ElevenLabs.

• ElevenLabsSTS: Template for transforming a voice into a different character or style using the ElevenLabs Speech-to-Speech API.

• ElevenLabsTTS: Template for converting text into speech using ElevenLabs' voice models.

• ElevenLabsVoiceClone: Template for creating a synthetic copy of an existing voice using the ElevenLabs API.

• ElevenLabsVoiceGeneration: Template for generating custom synthetic voices based on user-provided descriptions.

For specific instructions and further details, see the README.md.

Sinapsis F5-TTS

This package provides a template for seamlessly integrating, configuring, and running text-to-speech (TTS) functionalities powered by F5TTS.

• F5TTSInference: Converts text to speech using the F5TTS model with voice cloning capabilities.

For specific instructions and further details, see the README.md.

Sinapsis Kokoro

This package provides a single template for integrating, configuring, and running text-to-speech (TTS) synthesis using the Kokoro 82M v1.0 model.

KokoroTTS: Converts text to speech using the Kokoro TTS model. The template processes text packets from the input container, generates corresponding audio using Kokoro, and adds the resulting audio packets to the container. For specific instructions and further details, see the README.md.

Sinapsis Zonos

This package provides a single template for integrating, configuring, and running text-to-speech (TTS) and voice cloning functionalities powered by Zonos.

• ZonosTTS: Template for converting text to speech or performing voice cloning based on the presence of an audio sample.​

For specific instructions and further details, see the README.md.

Sinapsis Orppheus-CPP

This package provides a template for seamlessly integrating, configuring, and running text-to-speech (TTS) functionalities powered by Orpheus-TTS.

• OrpheusTTS: Converts text to speech using the Orpheus TTS model with advanced neural voice synthesis. The template processes text packets from the input container, generates corresponding audio using Orpheus TTS, and adds the resulting audio packets to the container. Features graceful error handling for out-of-memory conditions

For specific instructions and further details, see the README.md.

Sinapsis Parakeet-TDT

This package provides a template for seamlessly integrating, configuring, and running speech-to-text (STT) functionalities powered by NVIDIA's Parakeet TDT model.

• ParakeetTDTInference: Converts speech to text using NVIDIA's Parakeet TDT 0.6B model. This template processes audio packets from the input container or specified file paths, performs transcription with optional timestamp prediction, and adds the resulting text packets to the container.

For specific instructions and further details, see the README.md.

🌐 Webapps

The webapps included in this project showcase the modularity of the templates, in this case for speech generation tasks.

[!IMPORTANT] To run the app you first need to clone this repository:

git clone [e-mail]:Sinapsis-ai/sinapsis-speech.git
cd sinapsis-speech

[!NOTE] If you'd like to enable external app sharing in Gradio, export GRADIO_SHARE_APP=True

[!IMPORTANT] Elevenlabs requires an API key to run any inference. To get started, visit the official website and create an account. If you already have an account, go to the API keys page to generate a token.

[!IMPORTANT] Set your env var using export ELEVENLABS_API_KEY='your-api-key'

[!IMPORTANT] F5-TTS requires a reference audio file for voice cloning. Make sure you have a reference audio file in the artifacts directory.

[!NOTE] Agent configuration can be changed through the AGENT_CONFIG_PATH env var. You can check the available configurations in each package configs folder.

🐳 Docker

IMPORTANT: This Docker image depends on the sinapsis-nvidia:base image. For detailed instructions, please refer to the Sinapsis README.

1. Build the sinapsis-speech image:

docker compose -f docker/compose.yaml build

2. Start the app container:

• For ElevenLabs:

docker compose -f docker/compose_apps.yaml up -d sinapsis-elevenlabs

• For F5-TTS:

docker compose -f docker/compose_apps.yaml up -d sinapsis-f5_tts

• For Kokoro:

docker compose -f docker/compose_apps.yaml up -d sinapsis-kokoro

• For Zonos:

docker compose -f docker/compose_apps.yaml up -d sinapsis-zonos

• For Orpheus-CPP:

docker compose -f docker/compose_apps.yaml up -d sinapsis-orpheus-tts

• For Parakeet:

docker compose -f docker/compose_apps.yaml up -d sinapsis-parakeet

3. Check the logs

• For ElevenLabs:

docker logs -f sinapsis-elevenlabs

• For F5-TTS:

docker logs -f sinapsis-f5tts

• For Kokoro:

docker logs -f sinapsis-kokoro

• For Zonos:

docker logs -f sinapsis-zonos

• For Orpheus-CPP:

docker logs -f sinapsis-orpheus-tts

• For Parakeet:

docker logs -f sinapsis-parakeet

4. The logs will display the URL to access the webapp, e.g.,::

Running on local URL:  http://127.0.0.1:7860

NOTE: The url may be different, check the output of logs. 5. To stop the app:

docker compose -f docker/compose_apps.yaml down

💻 UV

To run the webapp using the uv package manager, follow these steps:

[!IMPORTANT] If you're using sinapsis-orpheus-cpp, you need to export cuda environment variables:

export CMAKE_ARGS="-DGGML_CUDA=on"
export FORCE_CMAKE="1"
export CUDACXX=$(command -v nvcc)

1. Sync the virtual environment:

uv sync --frozen

2. Install the wheel:

uv pip install sinapsis-speech[all] --extra-index-url https://pypi.sinapsis.tech

3. Run the webapp:

• For ElevenLabs:

uv run webapps/generic_tts_apps/elevenlabs_tts_app.py

• For F5-TTS:

uv run webapps/packet_tts_apps/f5_tts_app.py

• For Kokoro:

uv run webapps/packet_tts_apps/kokoro_tts_app.py

• For Zonos:

uv run webapps/generic_tts_apps/zonos_tts_app.py

4. The terminal will display the URL to access the webapp (e.g.):

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.

📙 Documentation

Documentation is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.