PyHearingAI

The official library for transcribing audio conversations with accurate speaker identification.

Current Status

PyHearingAI follows Clean Architecture principles with a well-organized code structure. The library provides a complete pipeline for audio transcription with speaker diarization and supports multiple output formats.

Features

• Audio format conversion (supports mp3, wav, mp4, and more)
• Transcription pipeline powered by OpenAI Whisper
• Speaker diarization using Pyannote
• Speaker assignment using GPT-4o
• Support for multiple output formats:
  • TXT
  • JSON
  • SRT
  • VTT
  • Markdown
• Clean Architecture design for maintainability and extensibility
• End-to-end testing framework
• Progress tracking for long-running processes
• Comprehensive error handling
• Command-line interface

Requirements

• Python 3.8+
• FFmpeg for audio conversion
• API keys:
  • OpenAI API key (for Whisper transcription and GPT-4o speaker assignment)
  • Hugging Face API key (for Pyannote speaker diarization)

Installation

System Dependencies

First, install FFmpeg:

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt-get install ffmpeg

# Windows (using Chocolatey)
choco install ffmpeg

Using Poetry (Recommended)

poetry add pyhearingai

Using pip

pip install pyhearingai

API Key Setup

Set up your API keys as environment variables:

# In your terminal or .env file
export OPENAI_API_KEY=your_openai_api_key
export HUGGINGFACE_API_KEY=your_huggingface_api_key

Or in your Python code:

import os
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"
os.environ["HUGGINGFACE_API_KEY"] = "your_huggingface_api_key"

Quick Start

# Simple one-line usage
from pyhearingai import transcribe

# Process an audio file with default settings
result = transcribe("meeting.mp3")

# Print the full transcript with speaker labels
print(result.text)

# Save in different formats
result.save("transcript.txt")  # Plain text
result.save("transcript.json")  # JSON with segments, timestamps
result.save("transcript.srt")   # Subtitle format
result.save("transcript.md")    # Markdown format

Advanced Usage

Configuring the Transcription Process

from pyhearingai import transcribe

# Configure transcription with specific options
result = transcribe(
    "interview.mp3",
    transcriber="whisper_openai",  # Specify transcriber
    diarizer="pyannote",           # Specify diarizer
    verbose=True                   # Enable verbose output
)

Progress Tracking

def progress_callback(progress_info):
    stage = progress_info.get('stage', 'unknown')
    percent = progress_info.get('progress', 0) * 100
    print(f"Processing {stage}: {percent:.1f}% complete")

result = transcribe(
    "long_recording.mp3",
    progress_callback=progress_callback
)

Working with Results

# Access the segments
for segment in result.segments:
    print(f"Speaker {segment.speaker_id}: {segment.text}")
    print(f"Time: {segment.start:.2f}s - {segment.end:.2f}s")

# Available output formats
from pyhearingai import list_output_formatters, get_output_formatter

# List available formatters
formatters = list_output_formatters()  # ['txt', 'json', 'srt', 'vtt', 'md']

# Get a specific formatter and format output
json_formatter = get_output_formatter('json')
json_content = json_formatter.format(result)
with open("transcript.json", "w") as f:
    f.write(json_content)

Command Line Interface

PyHearingAI includes a command-line interface:

# Basic usage
transcribe meeting.mp3

# Specify output format
transcribe meeting.mp3 --output transcript.txt

# Configure models
transcribe meeting.mp3 --transcriber whisper-openai --diarizer pyannote --speaker-assigner gpt-4o

# Get help
transcribe --help

Testing

The library includes an end-to-end test that validates the complete pipeline:

# Install test dependencies
pip install -r requirements_test.txt

# Run the end-to-end test
python -m pytest tests/test_end_to_end.py -v

Repository

PyHearingAI is hosted on GitHub:

• https://github.com/MDGrey33/PyHearingAI

Architecture

PyHearingAI follows Clean Architecture principles, with clear separation of concerns:

• Core (Domain Layer): Contains domain models and business rules
• Application Layer: Implements use cases like transcription and speaker assignment
• Infrastructure Layer: Provides concrete implementations of interfaces (OpenAI Whisper, Pyannote, GPT-4o)
• Presentation Layer: Offers user interfaces (CLI, future REST API)

For more details on the solution design and architecture, see the documentation:

• Core Architecture
• Solution Design

Extending PyHearingAI

The library is designed for extensibility:

Custom Transcriber

from pyhearingai.extensions import register_transcriber
from pyhearingai.models import Transcriber

@register_transcriber("my-transcriber")
class MyTranscriber(Transcriber):
    def transcribe(self, audio_path, **kwargs):
        # Custom transcription logic
        return segments

Custom Diarizer

from pyhearingai.extensions import register_diarizer
from pyhearingai.models import Diarizer

@register_diarizer("my-diarizer")
class MyDiarizer(Diarizer):
    def diarize(self, audio_path, **kwargs):
        # Custom diarization logic
        return speaker_segments

Custom Speaker Assigner

from pyhearingai.extensions import register_speaker_assigner
from pyhearingai.models import SpeakerAssigner

@register_speaker_assigner("my-assigner")
class MySpeakerAssigner(SpeakerAssigner):
    def assign_speakers(self, transcript_segments, diarization_segments, **kwargs):
        # Custom speaker assignment logic
        return labeled_segments

Custom Output Format

from pyhearingai.extensions import register_output_formatter
from pyhearingai.models import OutputFormatter

@register_output_formatter("my-format")
class MyOutputFormatter(OutputFormatter):
    def format(self, result):
        # Custom formatting logic
        return formatted_output

Logging

Configure logging to control verbosity:

import logging
logging.basicConfig(level=logging.INFO)

# Set specific logger levels
logging.getLogger('pyhearingai.transcription').setLevel(logging.DEBUG)
logging.getLogger('pyhearingai.diarization').setLevel(logging.WARNING)

Directory Structure

The library creates the following directory structure for outputs:

content/
├── audio_conversion/    # Converted audio files
├── transcription/       # Transcription results
├── diarization/         # Speaker diarization results
└── speaker_assignment/  # Final output with speaker labels

Privacy and Data Handling

When using PyHearingAI, be aware that:

• Audio data is sent to third-party APIs (OpenAI and Hugging Face)
• OpenAI's data usage policies apply to audio sent for transcription
• Hugging Face's data usage policies apply to audio sent for diarization
• Consider data processing agreements when processing sensitive information

API Rate Limits and Quotas

Users should be aware of:

• OpenAI has rate limits for the Whisper API (requests per minute)
• GPT-4o has token limits per request and rate limits
• Hugging Face API may have usage quotas

Environment Variables

Required environment variables:

OPENAI_API_KEY=your_openai_api_key
HUGGINGFACE_API_KEY=your_huggingface_api_key

Optional environment variables:

PYHEARINGAI_DEFAULT_TRANSCRIBER=whisper-openai
PYHEARINGAI_DEFAULT_DIARIZER=pyannote
PYHEARINGAI_DEFAULT_SPEAKER_ASSIGNER=gpt-4o
PYHEARINGAI_OUTPUT_DIR=./content
PYHEARINGAI_LOG_LEVEL=INFO

License

Apache 2.0

Implemented Features

• Multiple output formats (TXT, JSON, SRT, VTT, Markdown)
• Transcription models:
  • OpenAI Whisper API (default)
• Diarization models:
  • Pyannote
• Speaker assignment models:
  • GPT-4o (using OpenAI API)

Features Under Development

• 🎛️ Extended Model Support:

  • Local Whisper models
  • Faster Whisper
  • Additional diarization models

• 🚀 Performance Features:

  • GPU Acceleration
  • Batch processing
  • Memory optimization

Contributing

We welcome contributions! Please check our GitHub repository for guidelines.

Acknowledgments

• OpenAI for the Whisper and GPT models
• Pyannote for the diarization technology
• The open-source community for various contributions