sinapsis-elevenlabs 0.2.4

Generate realistic voices from text, other speeches, and more using the Elevenlabs library.

Sinapsis Elevenlabs

Templates for advanced speech generation with Elevenlabs

🐍 Installation • 🚀 Features • 📚 Usage example • 🌐 Webapp • 📙 Documentation • 🔍 License

This Sinapsis Elevenlabs package offers a suite of templates and utilities designed for effortless integrating, configuration, and execution of text-to-speech (TTS), speech-to-speech (STS), voice cloning, and voice generation functionalities powered by ElevenLabs.

🐍 Installation

[!IMPORTANT] Sinapsis project requires Python 3.10 or higher.

Install using your preferred package manager. We strongly recommend using uv. To install uv, refer to the official documentation.

Install with uv:

  uv pip install sinapsis-elevenlabs --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

  pip install sinapsis-elevenlabs --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] Templates in each package may require additional dependencies. For development, we recommend installing the package with all the optional dependencies:

With uv:

  uv pip install sinapsis-elevenlabs[all] --extra-index-url https://pypi.sinapsis.tech

Or with raw pip:

  pip install sinapsis-elevenlabs[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

• ElevenLabsSTS: Template for transforming a voice into a different character or style using the ElevenLabs Speech-to-Speech API.

  Attributes

  • api_key(Optional): The API key used to authenticate with ElevenLabs' API. Although this parameter is optional in the function signature, the API key must be provided either through this argument or the ELEVENLABS_API_KEY environmental variable. If neither is provided, an error will be logged, and no speech will be generated.
  • model(Optional): The model identifier to use for speech synthesis (default: eleven_multilingual_sts_v2). Options: eleven_english_sts_v2, eleven_multilingual_sts_v2.
  • output_format(Optional): The output audio format and quality (default: mp3_44100_128). Options: mp3_22050_32, mp3_44100_32, mp3_44100_64, mp3_44100_96, mp3_44100_128, mp3_44100_192, pcm_16000, pcm_22050, pcm_24000, pcm_44100, ulaw_8000.
  • output_folder(Optional): The folder where generated audio files will be saved (default: SINAPSIS_CACHE_DIR/elevenlabs/ audios).
  • stream(Optional): If True, the audio is returned as a stream; otherwise, saved to a file (default: False).
  • streaming_latency(Optional): Latency optimization for streaming (default: None).
  • voice(Optional): Voice for speech synthesis. Can be a voice ID (str), name (str), or ElevenLabs voice object (Voice) (default: None).
  • voice_settings(Optional):Dictionary of voice control settings:
    • stability: Controls voice randomness and emotional range (range: 0.0 to 1.0).
    • similarity_boost: Adjusts how closely the voice matches the original (range: 0.0 to 1.0).
    • style: Amplifies the speaker’s style, consuming more resources (range: 0.0 to 1.0).
    • use_speaker_boost: Increases similarity to the speaker with higher computational cost (boolean: True or False).
    • speed: Adjusts speech speed (range: 0.7 to 1.2, default: 1.0).

• ElevenLabsTTS: Template for converting text into speech using ElevenLabs' voice models.

  Attributes

  • api_key(Optional): The API key used to authenticate with ElevenLabs' API. Although this parameter is optional in the function signature, the API key must be provided either through this argument or the ELEVENLABS_API_KEY environmental variable. If neither is provided, an error will be logged, and no speech will be generated.
  • model(Optional): The model identifier to use for speech synthesis (default: eleven_turbo_v2_5). Options: eleven_turbo_v2_5, eleven_multilingual_v2, eleven_turbo_v2, eleven_monolingual_v1, eleven_multilingual_v1.
  • output_format(Optional): The output audio format and quality (default: mp3_44100_128). Options: mp3_22050_32, mp3_44100_32, mp3_44100_64, mp3_44100_96, mp3_44100_128, mp3_44100_192, pcm_16000, pcm_22050, pcm_24000, pcm_44100, ulaw_8000.
  • output_folder(Optional): The folder where generated audio files will be saved (default: SINAPSIS_CACHE_DIR/elevenlabs/audios).
  • stream(Optional): If True, the audio is returned as a stream; otherwise, saved to a file (default: False).
  • voice(Optional): Voice for speech synthesis. Can be a voice ID (str), name (str), or ElevenLabs voice object (Voice) (default: None).
  • voice_settings(Optional):Dictionary of voice control settings:
    • stability: Controls voice randomness and emotional range (range: 0.0 to 1.0).
    • similarity_boost: Adjusts how closely the voice matches the original (range: 0.0 to 1.0).
    • style: Amplifies the speaker’s style, consuming more resources (range: 0.0 to 1.0).
    • use_speaker_boost: Increases similarity to the speaker with higher computational cost (boolean: True or False).
    • speed: Adjusts speech speed (range: 0.7 to 1.2, default: 1.0).

• ElevenLabsVoiceClone: Template for creating a synthetic copy of an existing voice using the ElevenLabs API.

  Attributes

  • api_key(Optional): The API key used to authenticate with ElevenLabs' API. Although this parameter is optional in the function signature, the API key must be provided either through this argument or the ELEVENLABS_API_KEY environmental variable. If neither is provided, an error will be logged, and no speech will be generated.
  • description(Optional): Description for the cloned voice (default: None).
  • model(Optional): The model identifier to use for speech synthesis (default: eleven_turbo_v2_5). Options: eleven_turbo_v2_5, eleven_multilingual_v2, eleven_turbo_v2, eleven_monolingual_v1, eleven_multilingual_v1.
  • name(Optional): Name for the cloned voice (default: None). If None, a default name may be used.
  • output_format(Optional): The output audio format and quality (default: mp3_44100_128). Options: mp3_22050_32, mp3_44100_32, mp3_44100_64, mp3_44100_96, mp3_44100_128, mp3_44100_192, pcm_16000, pcm_22050, pcm_24000, pcm_44100, ulaw_8000.
  • output_folder(Optional): The folder where generated audio files will be saved (default: SINAPSIS_CACHE_DIR/elevenlabs/audios).
  • remove_background_noise(Optional): Whether to remove background noise from samples (default: False).
  • stream(Optional): If True, the audio is returned as a stream; otherwise, saved to a file (default: False).
  • voice(Optional): Voice for speech synthesis. Can be a voice ID (str), name (str), or ElevenLabs voice object (Voice) (default: None).
  • voice_description(Required): A description of the voice to be used for synthesis. This field is mandatory and helps to define the voice's characteristics or style.
  • voice_settings(Optional):Dictionary of voice control settings:
    • stability: Controls voice randomness and emotional range (range: 0.0 to 1.0).
    • similarity_boost: Adjusts how closely the voice matches the original (range: 0.0 to 1.0).
    • style: Amplifies the speaker’s style, consuming more resources (range: 0.0 to 1.0).
    • use_speaker_boost: Increases similarity to the speaker with higher computational cost (boolean: True or False).
    • speed: Adjusts speech speed (range: 0.7 to 1.2, default: 1.0).

• ElevenLabsVoiceGeneration: Template for generating custom synthetic voices based on user-provided descriptions.

  Attributes

  • api_key(Optional): The API key used to authenticate with ElevenLabs' API. Although this parameter is optional in the function signature, the API key must be provided either through this argument or the ELEVENLABS_API_KEY environmental variable. If neither is provided, an error will be logged, and no speech will be generated.
  • model(Optional): The model identifier to use for speech synthesis (default: eleven_turbo_v2_5). Options: eleven_turbo_v2_5, eleven_multilingual_v2, eleven_turbo_v2, eleven_monolingual_v1, eleven_multilingual_v1.
  • output_format(Optional): The output audio format and quality (default: mp3_44100_128). Options: mp3_22050_32, mp3_44100_32, mp3_44100_64, mp3_44100_96, mp3_44100_128, mp3_44100_192, pcm_16000, pcm_22050, pcm_24000, pcm_44100, ulaw_8000.
  • output_folder(Optional): The folder where generated audio files will be saved (default: SINAPSIS_CACHE_DIR/elevenlabs/audios).
  • stream(Optional): If True, the audio is returned as a stream; otherwise, saved to a file (default: False).
  • voice(Optional): Voice for speech synthesis. Can be a voice ID (str), name (str), or ElevenLabs voice object (Voice) (default: None).
  • voice_description(Required): A description of the voice to be used for synthesis. This field is mandatory and helps to define the voice's characteristics or style.
  • voice_settings(Optional):Dictionary of voice control settings:
    • stability: Controls voice randomness and emotional range (range: 0.0 to 1.0).
    • similarity_boost: Adjusts how closely the voice matches the original (range: 0.0 to 1.0).
    • style: Amplifies the speaker’s style, consuming more resources (range: 0.0 to 1.0).
    • use_speaker_boost: Increases similarity to the speaker with higher computational cost (boolean: True or False).
    • speed: Adjusts speech speed (range: 0.7 to 1.2, default: 1.0).

[!TIP] Use CLI command sinapsis info --example-template-config TEMPLATE_NAME to produce an example Agent config for the Template specified in TEMPLATE_NAME.

For example, for ElevenLabsTTS use sinapsis info --example-template-config ElevenLabsTTS to produce an example config like:

agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: ElevenLabsTTS
  class_name: ElevenLabsTTS
  template_input: InputTemplate
  attributes:
    api_key: null
    voice: null
    voice_settings:
      stability: null
      similarity_boost: null
      style: null
      use_speaker_boost: null
      speed: null
    model: eleven_turbo_v2_5
    output_format: mp3_44100_128
    output_folder: ~/.cache/sinapsis/elevenlabs/audios
    stream: false

📚 Usage example

This example shows how to use the ElevenLabsVoiceGeneration template to convert text into speech. It generates speech based on a voice created from the user's description and saves the resulting audio file locally.

Config

agent:
  name: voice_creation
  description: voice generation agent using Elevenlabs

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: TextInput
  class_name: TextInput
  template_input: InputTemplate
  attributes:
    text: En la oscuridad de la noche, se escuchaban los llantos lejanos de una mujer. Nadie sabía exactamente de dónde venían, pero todos los habitantes del pueblo aseguraban que era el llanto de La Llorona. Se decía que era el espíritu de una mujer que, en vida, había perdido a sus hijos y que, condenada por su dolor y su culpa, deambulaba por las orillas de los ríos buscando a sus pequeños. Nadie se atrevía a acercarse al agua cuando oían su llanto, pues sabían que, si la escuchabas cerca, su destino también estaba sellado...

- template_name: ElevenLabsVoiceGeneration
  class_name: ElevenLabsVoiceGeneration
  template_input: TextInput
  attributes:
    voice_description: A warm and engaging Mexican Spanish female voice, perfect for storytelling, audiobooks, and podcasts. Clear and expressive, with a natural, captivating tone, ideal for social media, YouTube, TikTok, and more.

This configuration defines an agent and a sequence of templates for speech synthesis, using a custom voice created through ElevenLabs.

[!IMPORTANT] The TextInput template correspond to sinapsis-data-readers. If you want to use the example, please make sure you install the package.

To run the config, use the CLI:

sinapsis run name_of_config.yml

🌐 Webapp

The webapps included in this project showcase the modularity of the ElevenLabs templates, in this case for speech generation tasks.

[!IMPORTANT] To run the app you first need to clone this repository:

git clone [e-mail]:Sinapsis-ai/sinapsis-speech.git
cd sinapsis-speech

[!NOTE] If you'd like to enable external app sharing in Gradio, export GRADIO_SHARE_APP=True

[!IMPORTANT] Elevenlabs requires an API key to run any inference. To get started, visit the official website and create an account. If you already have an account, go to the API keys page to generate a token.

[!IMPORTANT] Set your env var using export ELEVENLABS_API_KEY='your-api-key'

🐳 Docker

IMPORTANT This docker image depends on the sinapsis-nvidia:base image. Please refer to the official sinapsis instructions to Build with Docker.

1. Build the sinapsis-speech image:

docker compose -f docker/compose.yaml build

2. Start the app container:

docker compose -f docker/compose_apps.yaml up -d sinapsis-elevenlabs

3. Check the logs

docker logs -f sinapsis-elevenlabs

4. The logs will display the URL to access the webapp, e.g.,::

Running on local URL:  http://127.0.0.1:7860

NOTE: The url may be different, check the output of logs.

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

💻 UV

To run the webapp using the uv package manager, follow these steps:

1. Sync the virtual environment:

uv sync --frozen

2. Install the wheel:

uv pip install sinapsis-speech[all] --extra-index-url https://pypi.sinapsis.tech

3. Run the webapp:

uv run webapps/generic_tts_apps/elevenlabs_tts_app.py

4. The terminal will display the URL to access the webapp (e.g.):

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.

📙 Documentation

Documentation is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.