sinapsis-parakeet-tdt 0.1.5

Speech to text using Parakeet TDT model

Sinapsis Parakeet TDT

Templates for advanced speech-to-text transcription with NVIDIA Parakeet TDT

🐍 Installation • 🚀 Features • 📚 Usage example • 🌐 Webapp • 📙 Documentation • 🔍 License

This Sinapsis Parakeet TDT package provides a template for seamlessly integrating, configuring, and running speech-to-text (STT) functionalities powered by NVIDIA's Parakeet TDT model.

🐍 Installation

Install using your favourite package manager. We strongly encourage the use of uv, although any other package manager should work too. If you need to install uv please see the official documentation.

Example with uv:

  uv pip install sinapsis-parkeet-tdt --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-parkeet-tdt --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] Templates in each package may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

with uv:

  uv pip install sinapsis-parkeet-tdt[all] --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-parkeet-tdt[all] --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

This module includes a template for speech-to-text transcription using the Parakeet TDT model:

ParakeetTDTInference: Advanced speech-to-text transcription template powered by NVIDIA's Parakeet TDT 0.6B model, designed for high-quality English transcription with exceptional accuracy across a wide array of accents and dialects, vocal ranges, and diverse domains and noise conditions. The template features comprehensive support for punctuation, capitalization, and accurate timestamp prediction, ensuring professional-grade transcription results. It seamlessly processes audio packets from the input container or specified file paths.

Attributes

• model_name (str): Name or path of the Parakeet TDT model. Defaults to "nvidia/parakeet-tdt-0.6b-v2".
• audio_paths (list[str] | None): Optional list of audio file paths to transcribe. If None, audio will be taken from the AudioPackets in the DataContainer. Defaults to None.
• enable_timestamps (bool): Whether to generate timestamps for the transcription. Defaults to False.
• timestamp_level (Literal["char", "word", "segment"]): Level of timestamp detail. Defaults to "word".
• device (Literal["cpu", "cuda"]): Device to run the model on. Defaults to "cuda".
• refresh_cache (bool): Whether to refresh the cache when downloading the model. Defaults to False.

[!TIP] Use CLI command sinapsis info --example-template-config TEMPLATE_NAME to produce an example Agent config for the Template specified in TEMPLATE_NAME.

For example, for ParakeetTDTInference use sinapsis info --example-template-config ParakeetTDTInference to produce an example config like:

agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: ParakeetTDTInference
  class_name: ParakeetTDTInference
  template_input: InputTemplate
  attributes:
    model_name: "nvidia/parakeet-tdt-0.6b-v2"
    audio_paths: []
    enable_timestamps: false
    timestamp_level: "word"
    device: "cuda"
    refresh_cache: false

📚 Usage example

This example illustrates how to use the ParakeetTDTInference template for speech-to-text transcription. It converts audio input into text using NVIDIA's Parakeet TDT model.

Config

agent:
  name: parakeet_tdt_agent
  description: "Agent that transcribes speech to text using the NVIDIA Parakeet TDT model."

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: AudioReaderSoundfile
  class_name: AudioReaderSoundfile
  template_input: InputTemplate
  attributes:
    audio_file_path: "artifacts/sample.wav"
    source: "artifacts/sample.wav"

- template_name: ParakeetTDTInference
  class_name: ParakeetTDTInference
  template_input: AudioReaderSoundfile
  attributes:
    model_name: "nvidia/parakeet-tdt-0.6b-v2"
    enable_timestamps: true
    timestamp_level: "word"
    device: "cuda"

This configuration defines a complete pipeline for speech-to-text transcription:

1. First, an audio file is read using the AudioReaderSoundfile template
2. The audio is then processed by the ParakeetTDTInference template, which transcribes it to text
3. The transcription is saved to a text file using the TextWriter template

[!IMPORTANT] The AudioReaderSoundfile and TextWriter templates correspond to sinapsis-data-readers. If you want to use the example, please make sure you install these packages.

To run the config, use the CLI:

sinapsis run name_of_config.yml

🌐 Webapp

The webapp included in this project showcases the capabilities of the Parakeet TDT model for speech recognition tasks.

[!IMPORTANT] To run the app you first need to clone this repository:

git clone [e-mail]:Sinapsis-ai/sinapsis-speech.git
cd sinapsis-speech

[!NOTE] If you'd like to enable external app sharing in Gradio, export GRADIO_SHARE_APP=True

🐳 Docker

IMPORTANT This docker image depends on the sinapsis-nvidia:base image. Please refer to the official sinapsis instructions to Build with Docker.

1. Build the sinapsis-speech image:

docker compose -f docker/compose.yaml build

2. Start the app container:

docker compose -f docker/compose_apps.yaml up -d sinapsis-parakeet-tdt

3. Check the logs

docker logs -f sinapsis-parakeet-tdt

4. The logs will display the URL to access the webapp, e.g.,::

Running on local URL:  http://127.0.0.1:7860

NOTE: The url may be different, check the output of logs.

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

💻 UV

To run the webapp using the uv package manager, follow these steps:

1. Sync the virtual environment:

uv sync --frozen

2. Install the wheel:

uv pip install sinapsis-speech[all] --extra-index-url https://pypi.sinapsis.tech

3. Run the webapp:

uv run webapps/speech_to_text_apps/parakeet_tdt_app.py

4. The terminal will display the URL to access the webapp (e.g.):

Running on local URL:  http://127.0.0.1:7860

NOTE: The URL may vary; check the terminal output for the correct address.

📙 Documentation

Documentation is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.