• n_gpu_layers: Number of model layers to offload to GPU (-1 = use all layers, 0 = CPU only) (default: -1)
• n_threads: Number of CPU threads to use for model inference (0 = auto-detect) (default: 0)
• n_ctx: Context window size (maximum number of tokens, 0 = use model's maximum) (default: 8192)
• model_id: Hugging Face model repository ID (required)
• model_variant: Specific GGUF file to download from the repository (default: None)
• cache_dir: Directory to store downloaded models and cache files (default: SINAPSIS_CACHE_DIR)
• verbose: Enable verbose logging for model operations (default: False)
• voice_id: Voice identifier for speech synthesis (required)
• batch_size: Batch size for model inference (default: 1)
• max_tokens: Maximum number of tokens to generate for speech (default: 2048)
• temperature: Sampling temperature for token generation (default: 0.8)
• top_p: Nucleus sampling probability threshold (default: 0.95)
• top_k: Top-k sampling parameter (default: 40)
• min_p: Minimum probability threshold for token selection (default: 0.05)
• pre_buffer_size: Duration in seconds of audio to generate before yielding the first chunk (default: 1.5)

agent:
  name: orpheus_tts_agent
  description: "Agent that generates speech from text using the Orpheus TTS model."

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: TextInput
  class_name: TextInput
  template_input: InputTemplate
  attributes:
    source: "user_input"
    text: "Hi, I'm Tara. Welcome to Orpheus text-to-speech system! I can speak in a very natural way."

- template_name: OrpheusTTS
  class_name: OrpheusTTS
  template_input: TextInput
  attributes:
    n_gpu_layers: -1
    n_ctx: 4096
    model_id: "isaiahbjork/orpheus-3b-0.1-ft-Q4_K_M-GGUF"
    voice_id: "tara"
    temperature: 0.8
    top_p: 0.95
    top_k: 40
    min_p: 0.05
    pre_buffer_size: 1.5
    max_tokens: 2048

- template_name: SaveGeneratedAudio
  class_name: AudioWriterSoundfile
  template_input: OrpheusTTS
  attributes:
    save_dir: "orpheus_tts"
    root_dir: "artifacts"
    extension: "wav"

IMPORTANT This docker image depends on the sinapsis-nvidia:base image. Please refer to the official sinapsis instructions to Build with Docker.

1. Build the sinapsis-speech image:

docker compose -f docker/compose.yaml build

2. Start the app container:

docker compose -f docker/compose_apps.yaml up -d sinapsis-orpheus-tts

3. Check the logs

docker logs -f sinapsis-orpheus-tts

4. The logs will display the URL to access the webapp, e.g.,::

Running on local URL:  http://127.0.0.1:7860

NOTE: The url may be different, check the output of logs.

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

To run the webapp using the uv package manager, follow these steps:

1. Export the environment variable to install the python bindings for llama-cpp:

export CMAKE_ARGS="-DGGML_CUDA=on"
export FORCE_CMAKE="1"

2. Export CUDACXX:

export CUDACXX=$(command -v nvcc)

3. Sync the virtual environment:

uv sync --frozen

4. Install the wheel:

uv pip install sinapsis-speech[all] --extra-index-url https://pypi.sinapsis.tech

5. Run the webapp:

uv run webapps/packet_tts_apps/orpheus_tts_app.py

6. The terminal will display the URL to access the webapp (e.g.):

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.