Skip to main content

Evaluating ASR (automatic speech recognition) hypotheses, i.e. computing word error rate.

Project description


Build Status PyPI version codecov

Python module for evaluting ASR hypotheses (i.e. word error rate and word recognition rate).

This module depends on the editdistance project, for computing edit distances between arbitrary sequences.

The formatting of the output of this program is very loosely based around the same idea as the align.c program commonly used within the Sphinx ASR community. This may run a bit faster if neither instances nor confusions are printed.

Please let me know if you have any comments, questions, or problems.


The program outputs three standard measurements:

  • Word error rate (WER)
  • Word recognition rate (the number of matched words in the alignment divided by the number of words in the reference).
  • Sentence error rate (SER) (the number of incorrect sentences divided by the total number of sentences).

Installing & uninstalling

The easiest way to install is using pip:

pip install asr-evaluation

Alternatively you can clone this git repo and install using distutils:

git clone
cd asr-evaluation
python install

To uninstall with pip:

pip uninstall asr-evaluation

Command line usage

For command line usage, see:

    wer --help

It should display something like this:

usage: wer [-h] [-i | -r] [--head-ids] [-id] [-c] [-p] [-m count] [-a] [-e]
           ref hyp

Evaluate an ASR transcript against a reference transcript.

positional arguments:
  ref                   Reference transcript filename
  hyp                   ASR hypothesis filename

optional arguments:
  -h, --help            show this help message and exit
  -i, --print-instances
                        Print all individual sentences and their errors.
  -r, --print-errors    Print all individual sentences that contain errors.
  --head-ids            Hypothesis and reference files have ids in the first
                        token? (Kaldi format)
  -id, --tail-ids, --has-ids
                        Hypothesis and reference files have ids in the last
                        token? (Sphinx format)
  -c, --confusions      Print tables of which words were confused.
  -p, --print-wer-vs-length
                        Print table of average WER grouped by reference
                        sentence length.
  -m count, --min-word-count count
                        Minimum word count to show a word in confusions.
  -a, --case-insensitive
                        Down-case the text before running the evaluation.
  -e, --remove-empty-refs
                        Skip over any examples where the reference is empty.

Contributing and code of conduct

For contributions, it's best to Github issues and pull requests. Proper testing and documentation suggested.

Code of conduct is expected to be reasonable, especially as specified by the Contributor Covenant

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

asr_evaluation-2.0.4.tar.gz (8.2 kB view hashes)

Uploaded source

Built Distribution

asr_evaluation-2.0.4-py3-none-any.whl (9.1 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page