whisper-realtime-transcriber 0.1.2

Let whisper models transcribe in realtime.

Whisper Realtime Transcriber

Overview

This repository contains the source code of a realtime transcriber for various whisper models, published on huggingface.

Prerequisites

Before you begin, make sure you meet the following prerequisites:

• Python 3.10.12 installed on your machine.
• Microphone connected to your machine.

Installation

1. Install torch with CUDA support (optional)

• Follow the instructions here and install version >=2.4.0

2. Install the package:

   pip install --upgrade whisper-realtime-transcriber
   

Usage

After completing the installation, you can now use the transcriber:

# Necessary imports
import asyncio

from whisper_realtime_transcriber.InputStreamGenerator import InputStreamGenerator
from whisper_realtime_transcriber.WhisperModel import WhisperModel
from whisper_realtime_transcriber.RealtimeTranscriber import RealtimeTranscriber

# Standard way - all outputs get printed directly to the console.
inputstream_generator = InputStreamGenerator()
asr_model = WhisperModel(inputstream_generator)

transcriber = RealtimeTranscriber(inputstream_generator, asr_model)

asyncio.run(transcriber.execute_event_loop())

# Loading a custom model id
# When specifying model_id, model_size becomes obsolete
inputstream_generator = InputStreamGenerator()
asr_model = WhisperModel(inputstream_generator, model_id="openai/whisper-tiny")
# ... #

# Executing a custom function inside the RealtimeTranscriber.
def print_transcription(some_transcription):
  print(some_transcription)

inputstream_generator = InputStreamGenerator()
asr_model = WhisperModel(inputstream_generator)

# specifying a function and setting continuous to False will allow one
# to execute a custom function during the event loop, that is doing something with the transcriptions
transcriber = RealtimeTranscriber(inputstream_generator, asr_model, continuous=False, func=print_transcription)
  
asyncio.run(transcriber.execute_event_loop())

Feel free to reach out if you encounter any issues or have questions!

How it works

• The transcriber consists of two modules: a Inputstream Generator and a Whisper Model.
• The implementation of the Inputstream Generator is based on this implemantation.
• The Inputstream Generator reads the microphone input and passes it to the Whisper Model. The Whisper Model then generates the transcription.
• This is happening in an async event loop so that the Whsiper Model can continuously generate transcriptions from the provided audio input, generated and processed by the Inputstream Generator.
• On a machine with a 12GB Nvidia RTX 3060 the distilled large-v3 model runs at a realtime-factor of about 0.4, this means 10s of audio input get transcribed in 4s - the longer the input the bigger is the realtime-factor.

ToDos

• Add functionality to transcribe from audio files.
• Get somehow rid of the hallucinations of the whisper models when no voice is active.