Skip to main content


Sinapsis
Sinapsis Diarization

Combine speaker diarization, ASR and optional emotion signals.

Overview

sinapsis-diarization — Speech processing templates and pipelines for transcription, speaker diarization, and emotion analysis.

A transcript without speaker attribution loses who said what. These Templates support workflows that align speech recognition with speaker segments and, optionally, additional audio analytics.

Where it fits

The implementation offers multiple backend choices with different GPU and dependency needs. pyannote, whisperx and emotion extras select the relevant capabilities; not every task needs all of them.

Installation

Requires Python 3.13 or later, as declared in this package’s manifest. Install the base distribution using your preferred package manager:

uv pip install sinapsis-diarization
# or
pip install sinapsis-diarization

Optional features and dependencies

The base installation includes the dependencies required by the package. The following extras are declared in this package’s pyproject.toml; install only those needed by your workflow. They are standard Python optional dependencies, not separate execution profiles.

Extra Adds / when to select it
pyannote pyannote speaker diarization backend. Dependencies: pyannote-audio, torchcodec
data-tools Media readers/writers or data utilities for a complete pipeline. Dependencies: sinapsis-data-readers
all Bundle declared by this package; its manifest includes data-tools, dev, emotion, pyannote, whisperx.
emotion Optional emotion analysis functionality. Dependencies: ruamel-yaml, speechbrain
whisperx WhisperX transcription/alignment backend. Dependencies: onnx, onnxruntime, whisperx
dev Development-only tooling, not a runtime feature. Dependencies: ty

For example:

uv pip install "sinapsis-diarization[pyannote]"

The manifest also defines development-only dependency groups: dev (pre-commit, ruff, ty). These support source development and testing and are not runtime extras.

🚀 Features

Templates

The Sinapsis Diarization module provides multiple templates for Automatic Speech Recognition, Diarization, Emotion Recognition and combined pipelines.

Automatic Speech Recognition (ASR)

These templates provide transcriptions of audios.

  • SinapsisParakeetASR: Runs Parakeet speech recognition for audio transcription.
  • SinapsisCanaryASR: Runs Canary speech recognition for audio transcription

Diarization

These templates identify speakers in a conversation and their turn times during the audio.

  • SinapsisSortformerDiarizer: Runs Sortformer to get diarization of the speakers in an audio.
  • SinapsisPyannoteDiarizer: Runs Pyannote to get diarization of speakers in an audio.

ASR plus Diarization

These templates combine ASR with Diarization to transcribe a conversation and identify the speakers automatically.

  • PyannoteDiarizedTranscription: Runs Parakeet and Pyannote to transcribe a conversation with speaker identification.
  • SortformerDiarizedTranscription: Runs Parakeet and Sortformer to transcribe a conversation with speaker identification.
  • WhisperxDiarizedTranscription: Runs Whisperx to transcribe an audio and divide it by speaker and time.

ASR plus Diarization and Emotion Recognition

These templates combine the diarized transcription templates with an emotion recognition pipeline to show the emotions detected by each speaker in the audio.

  • PyannoteEmotionTranscription: Runs Parakeet, Pyannote and Speechbrain to transcribe an audio, divide it by speaker and assign emotions to the segments.
  • SortformerEmotionTranscription: Runs Parakeet, Sortformer and Speechbrain to transcribe an audio, divide it by speaker and assign emotions to the segments.

Examples for template use can be found at src/sinapsis_diarization/configs

To get information on any of the templates use uv run sinx templates docstring TemplateName and substitute TemplateName for any of the sinapsis-diarization templates

📚 Example usage

All templates can be run through YAML configuration files using the sinx jobs run command. The example below shows how to set up the WhisperxDiarizedTranscription template, which performs ASR and speaker diarization in a single pipeline.

Configuration attributes:

  • root_dir: Base directory for resolving relative file paths.
  • audio_file_path: Path to the audio file to process (relative to root_dir).
  • asr_model_name: WhisperX model size. Available options: "tiny", "base", "small", "medium", "large-v1", "large-v2", "large-v3".
  • device: "cuda" or "cpu".
  • chunk_size_in_secs: Split audio into chunks of n seconds for processing. Use -1 to process the full audio at once.
  • min_speakers / max_speakers: Expected speaker count range.
Config file
agent:
  name: whisperx_asr_diarization
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: WhisperxDiarizedTranscription
  class_name: WhisperxDiarizedTranscription
  template_input: InputTemplate
  attributes:
    root_dir: path_to_root_dir
    audio_file_path: audio_sample.mp3
    asr_model_name: large-v3
    device: cuda
    sample_rate: 16000
    chunk_size_in_secs: 20
    min_speakers: 2
    max_speakers: 2

To run the agent with the config above:

sinx jobs run /path/to/sinapsis-diarization/src/sinapsis_diarization/configs/transcription_with_diarization/whisperx_asr_diarization.yaml

More config examples for all available templates can be found in src/sinapsis_diarization/configs/.

📙 CLI

The pipelines for ASR, Diarization and Emotion diarization are available as CLI commands. Each command takes an audio file and model options as input, then writes results to a given output directory (by default results). Required arguments are marked with (required).

Sinapsis ASR

Transcribe an audio file using a single ASR engine.

Run using uv run sinapsis-asr

Example: uv run sinapsis-asr --audio "path to audio" --model parakeet --chunk-size-in-secs 20 --device cuda

This command has the following options:

--audio AUDIO_PATH       Path to audio (required)
--model MODEL            Type of model to run (required)
--device DEVICE          Device to run the model (required)
--chunk-size-in-secs CHUNK_SIZE Size of chunks in seconds
--model-name MODEL_NAME  Name of model to use
--sample-rate SAMPLE_RATE Sample rate of audio
--output-dir OUTPUT_DIR  Output directory for results

Models

  • "parakeet"
  • "canary"

Model names

  • Parakeet:

    • "nvidia/parakeet-tdt-0.6b-v2"
    • "nvidia/parakeet-tdt-0.6b-v3"
  • Canary:

    • "nvidia/canary-qwen-2.5b"

Device options

  • "cuda"
  • "cpu"
Sinapsis Diarize

Identify speakers and their turn times in an audio file using a single diarization engine.

Run using uv run sinapsis-diarize

Example: uv run sinapsis-diarize --audio "path to audio" --model sortformer --chunk-size-in-secs 20 --device cuda

This command has the following options:

--audio AUDIO_PATH       Path to audio (required)
--model MODEL            Type of model to run (required)
--device DEVICE          Device to run the model (required)
--chunk-size-in-secs CHUNK_SIZE Size of chunks in seconds
--model-name MODEL_NAME  Name of model to use
--sample-rate SAMPLE_RATE Sample rate of audio
--output-dir OUTPUT_DIR  Output directory for results

Models

  • "sortformer"
  • "pyannote"

Model names

  • Sortformer:

    • "nvidia/diar_streaming_sortformer_4spk-v2.1"
  • Pyannote:

    • "pyannote/speaker-diarization-community-1"

Device options

  • "cuda"
  • "cpu"
Sinapsis ASR Diarize

Transcribe and diarize an audio file by composing separate ASR and diarization engines. Produces a speaker-attributed transcript.

Run using uv run sinapsis-asr-diarize

Example:

uv run sinapsis-asr-diarize --audio "path to audio" --asr-model parakeet --diarization-model sortformer --chunk-size-in-secs 20 --device cuda

This command has the following options:

--audio AUDIO_PATH       Path to audio (required)
--asr-model ASR_MODEL    Type of ASR model to run (required)
--diarization-model DIARIZATION_MODEL Type of Diarization model to run (required)
--device DEVICE          Device to run the model (required)
--chunk-size-in-secs CHUNK_SIZE Size of chunks in seconds
--asr-model-name ASR_MODEL_NAME Name of ASR model to use
--diarization-model-name DIARIZATION_MODEL_NAME Name of Diarization model to use
--sample-rate SAMPLE_RATE Sample rate of audio
--output-dir OUTPUT_DIR  Output directory for results
--num-speakers NUM_SPEAKERS Number of speakers for models that require it

ASR Models

  • "parakeet"
  • "canary"

Diarization Models

  • "sortformer"
  • "pyannote"

Model names

  • Parakeet:

    • "nvidia/parakeet-tdt-0.6b-v2"
    • "nvidia/parakeet-tdt-0.6b-v3"
  • Canary:

    • "nvidia/canary-qwen-2.5b"
  • Sortformer:

    • "nvidia/diar_streaming_sortformer_4spk-v2.1"
  • Pyannote:

    • "pyannote/speaker-diarization-community-1"

Device options

  • "cuda"
  • "cpu"
Sinapsis Whisperx ASR Diarize

Transcribe and diarize an audio file using the WhisperX monolithic pipeline. Unlike sinapsis-asr-diarize, this uses a single model for both ASR and diarization.

Run using uv run sinapsis-whisperx-asr-diarize

Example:

uv run sinapsis-whisperx-asr-diarize --audio "path to audio" --model-name large-v3 --chunk-size-in-secs 20 --device cuda

This command has the following options:

--audio AUDIO_PATH       Path to audio (required)
--model-name MODEL       WhisperX model size (required)
--device DEVICE          Device to run the model (required)
--chunk-size-in-secs CHUNK_SIZE Size of chunks in seconds
--sample-rate SAMPLE_RATE Sample rate of audio
--output-dir OUTPUT_DIR  Output directory for results
--min-speakers MIN_SPEAKERS Minimum number of speakers
--max-speakers MAX_SPEAKERS Maximum number of speakers

Model names

  • "tiny"
  • "base"
  • "small"
  • "medium"
  • "large-v1"
  • "large-v2"
  • "large-v3"

Device options

  • "cuda"
  • "cpu"
Sinapsis ASR Diarize Emotion

Transcribe, diarize, and detect emotions in an audio file. Composes ASR, diarization, and emotion detection engines to produce a speaker-attributed transcript with per-segment emotions.

Run using uv run sinapsis-asr-diarize-emotion

Example:

uv run sinapsis-asr-diarize-emotion --audio "path to audio" --asr-model parakeet --diarization-model sortformer --emotion-model speechbrain --chunk-size-in-secs 20 --device cuda

This command has the following options:

--audio AUDIO_PATH       Path to audio (required)
--asr-model ASR_MODEL    Type of ASR model to run (required)
--diarization-model DIARIZATION_MODEL Type of Diarization model to run (required)
--emotion-model EMOTION_MODEL Type of Emotion model to run (required)
--device DEVICE          Device to run the model (required)
--chunk-size-in-secs CHUNK_SIZE Size of chunks in seconds
--asr-model-name ASR_MODEL_NAME Name of ASR model to use
--diarization-model-name DIARIZATION_MODEL_NAME Name of Diarization model to use
--emotion-model-name EMOTION_MODEL_NAME Name of emotion model to use
--sample-rate SAMPLE_RATE Sample rate of audio
--output-dir OUTPUT_DIR  Output directory for results
--num-speakers NUM_SPEAKERS Number of speakers for models that require it

ASR Models

  • "parakeet"
  • "canary"

Diarization Models

  • "sortformer"
  • "pyannote"

Emotion Models

  • "speechbrain"

Model names

  • Parakeet:

    • "nvidia/parakeet-tdt-0.6b-v2"
    • "nvidia/parakeet-tdt-0.6b-v3"
  • Canary:

    • "nvidia/canary-qwen-2.5b"
  • Sortformer:

    • "nvidia/diar_streaming_sortformer_4spk-v2.1"
  • Pyannote:

    • "pyannote/speaker-diarization-community-1"

Device options

  • "cuda"
  • "cpu"

Example Agent

The following is adapted directly from the package’s shipped canary_asr.yaml configuration. Save it as my_diarization_agent.yaml. The example may require model checkpoints, input files, additional Template packages, or provider credentials as described in its configuration.

agent:
  name: my_diarization_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: SinapsisCanaryASR
  class_name: SinapsisCanaryASR
  template_input: InputTemplate
  attributes:
    model_name: "nvidia/canary-qwen-2.5b"
    device: cuda
    chunk_size_in_secs: -1
    root_dir: path_to_root_dir
    audio_file_path: audio_sample.mp3
sinx jobs run my_diarization_agent.yaml

More configurations: canary_asr.yaml, parakeet_asr.yaml, sortformer_diarizer.yaml, pyannote_diarizer.yaml, parakeet_sortformer_pipeline.yaml, parakeet_pyannote_pipeline.yaml, whisperx_asr_diarization.yaml, parakeet_sortformer_emotion.yaml, canary_asr_packet.yaml, parakeet_asr_packet.yaml, parakeet_pyannote_emotion.yaml, sinapsis_parakeet_sortformer_worker.yaml, sinapsis_pyannote_emotion_worker.yaml, sinapsis_sortformer_emotion_worker.yaml (and 11 more).

Documentation

License

Sinapsis open-source components are available under your choice of the MIT License or the Apache License 2.0, unless otherwise noted.

Metadata

Release files for sinapsis-diarization 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sinapsis-diarization 1.0.0
File Size Uploaded Provenance Actions
sinapsis_diarization-1.0.0.tar.gz 340.2 kB Without provenance Details for sinapsis_diarization-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sinapsis-diarization 1.0.0
File Interpreter ABI Platform Actions
sinapsis_diarization-1.0.0-py3-none-any.whl Python 3 none any Details for sinapsis_diarization-1.0.0-py3-none-any.whl

Total release size: 411.8 kB

Release files / sinapsis_diarization-1.0.0.tar.gz

Download URL sinapsis_diarization-1.0.0.tar.gz
Size 340.2 kB
Tags Source
SHA-256 checksum
How to use checksums
30b30745b6e6e440b9892cf39dec99adb538d51a1cf4d39b3c085b94cc023be9
BLAKE2b-256 checksum
How to use checksums
db55f9abb347d8340446431c89f0d45892fc9aa45e2b7ccf93f1dbd080b25c22
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.17

Release files / sinapsis_diarization-1.0.0-py3-none-any.whl

Download URL sinapsis_diarization-1.0.0-py3-none-any.whl
Size 71.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d17c63605f4d8cd48cc427284341af44f804eee4640a4f12f14a77c4a6610410
BLAKE2b-256 checksum
How to use checksums
c18e2426d95f62121c39c9a6ea23d7af442584fed612fb6add0627f39df33c36
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.6.17

Security for sinapsis-diarization 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page