afaligner 0.1.0

A forced aligner intended for synchronization of narrated text

afaligner

Overview

afaligner is a Python library for automatic text and audio synchronization. It's a forced aligner that works by synthesizing text and then aligning synthesized and recorded audio using a variation of DTW (Dynamic Time Warping) algorithm.

afaligner is used in syncabook tool for synchronization of narrated audio with text – to produce EPUB3 with Media Overlays ebooks – and has been developed for this specific purpose. If you want to create an ebook with synchronized text and audio, please refer to syncabook.

The main features of the alignment algorithm behind afaligner are:

• It can handle structural differences in the beginning and in the end of files which is often the case with audiobooks (e.g. disclaimers).

• It finds an approximation to an optimal warping path in linear time and space using FastDTW approach which, combined with the fact that The alignment algorithm itself is written in C, makes it pretty fast compared to other forced aligners.

• It can, with varying success, align differently splitted text and audio.

afaligner was inspired by aeneas and works in a similiar way. It uses aeneas as a dependency for text synthesis and MFCC extraction.

Requirements

• Python (>= 3.6)
• FFmpeg
• eSpeak
• Python packages: aeneas, numpy, jinja2

Installation

1. Install Python (>= 3.6)

2. Install FFmpeg and eSpeak

3. Install numpy:

$ pip install numpy

4. Install afaligner:

$ pip install afaligner

Or if you want to modify afaligner's code:

1. Get afaligner:

$ git clone https://github.com/r4victor/afaligner/ && cd afaligner

2. Install afaligner in editable mode:

$ pip install -e .

Usage

afaligner is designed to be used as a library. If you want to produce an ebook with synchronized text and audio or just to perform a synchronization, you may want to take a look at a command line tool called syncabook.

afaligner provides only one function called align which takes text directory, audio directory and a set of output parameters and returns sync map (a mapping from text fragments to their time positions in audio files) and, if requested, writes the result to disk. The call may look like this:

from afaligner import align


sync_map = align(
    'ebooks/demoebook/text/',
    'ebooks/demoebook/audio/',
    output_dir='ebooks/demoebook/smil/',
    output_format='smil',
    sync_map_text_path_prefix='../text/',
    sync_map_audio_path_prefix='../audio/'
)

For more details, please refer to docstrings.