Skip to main content

vac_aligner

PyPI Version VAC - VAD-ASR-CER (Matching) Pipeline Documentation Status

Comprehensive pipeline designed for processing long audio recordings through three main stages: Voice Activity Detection (VAD), Automatic Speech Recognition (ASR), and Character Error Rate (CER) Matching. This pipeline is ideal for improving speech recognition models and training Text-to-Speech (TTS) systems with high accuracy.

Features

  • Robust alignment for long audio sequences

  • Support for multiple languages (mostly western)

Usage

The pipeline processes multi-hour long audio and texts to produce many multi-second audio chunks and corresponding texts with an accuracy of 97%. This high accuracy is achieved through sophisticated matching algorithms that correct common ASR errors such as repeated characters, incomplete words, and incorrect word predictions.

There are a couple of ways the library can be used. One can use the full functionality by

from vac_aligner import run_pipeline

run_pipeline(
   manifest_file="path/to/save/manifest.json", batch_size=64,
   asr_input_file="path/to/manifest.json", # or path to folder, containing audio chunks (.wav)s
   target_base="path/where/to/save/artifacts", # otherwise, will use `asr_input_file`
   init_aligner_after_asr=True  # If there is no long transcript available and you need to extract it
)

There are many scenarios where one might need a partial functionality of the pipeline. Then we can use the classes directly.

Scenario 1

Need to perform ASR over short chunks and store in nemo_manifest

import os

from vac_aligner.dto import ASRConfig
from vac_aligner.asr import ASR_MAPPING, ASR

language = "hy"  #  Armenain
model_name = ASR_MAPPING[language] # "Yeroyan/stt_arm_conformer_ctc_large"
asr_config = ASRConfig(hf_token=os.environ['HF_TOKEN'], batch_size=24)
asr = ASR(model_name, asr_config)
asr.run(
    save_dir="where/to/save/asr/predictions", # .txt(s)
    save_manifest="path/to/save/manifest.json",
    wav_files="path/to/wav/files",
    test_manifest='path/to/manifest/with/predictions.json'
)

Scenario 2

You already have predictions manifest and/or long transcript, and want to run the Matching to obtain the correct chunk texts to replace ASR predictions.

from vac_aligner.matching import ArmenianAlignerVAC

output_file = 'path/to/save/combined/transcript.txt'
predictions_manifest = 'path/to/manifest/with/predictions.json'
chunks, combined_transcript = ArmenianAlignerVAC.combine_transcript(predictions_manifest,
                                                                    output_file,
                                                                    ending_punctuations="․,։")

and then matching

from vac_aligner.matching import ArmenianAlignerVAC

target_base = 'path/to/save/artifacts'
matches_sorted = ArmenianAlignerVAC(combined_transcript,
                                    chunks, output_file.replace(".txt", ".json"),
                                    target_base=target_base).align(0.35)

and get the benchmark

from vac_aligner.matching.benchmark_on_mcv import Benchmark

benchmark = Benchmark(target_base, predictions_manifest)
stats = benchmark.get_benchmark()
benchmark.analyze_and_save_benchmark(stats, output_file)

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.0.2 (2024-06-01)

Fix template errors (dates, package name, etc) Add usage examples (partial components of the pipeline) in README.md

0.0.1 (2024-05-31)

  • First release on PyPI.

  1. Design library structure

  2. Implement 2 main steps of the VAC
    • ASR

    • CER based Matching

Metadata

Release files for vac-aligner 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for vac-aligner 0.2.0
File Size Uploaded Provenance Actions
vac_aligner-0.2.0.tar.gz 1.2 MB Without provenance Details for vac_aligner-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for vac-aligner 0.2.0
File Interpreter ABI Platform Actions
vac_aligner-0.2.0-py2.py3-none-any.whl Python 3, Python 2 none any Details for vac_aligner-0.2.0-py2.py3-none-any.whl

Total release size: 3.5 MB

Release files / vac_aligner-0.2.0.tar.gz

Download URL vac_aligner-0.2.0.tar.gz
Size 1.2 MB
Tags Source
SHA-256 checksum
How to use checksums
9b322169ad319e98d14c41be27be3670757ff49c597486c3d351c5984d5ceed7
BLAKE2b-256 checksum
How to use checksums
57918d88a8b2b51eec779f139e5107604c0c6a89a9d5090bbbbbe3b3044b1325
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.9.20

Release files / vac_aligner-0.2.0-py2.py3-none-any.whl

Download URL vac_aligner-0.2.0-py2.py3-none-any.whl
Size 2.4 MB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
30497620b5a38fb91f0a55f4abf7b5ac61541c78a555ea73dc645bc6ccafb78b
BLAKE2b-256 checksum
How to use checksums
430c2f7e88986680872625b8bd62698ffe0029316646c1f241fa413f1b93daf4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.9.20

Security for vac-aligner 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page