openspeechcorpus 0.2.1

The CLI for perform actions over the Open Speech Corpus

Open Speech Corpus CLI

This repository contains the code required to download audiodata from openspeechcorpus.com

Open Speech Corpus is composed by far for three subcorpuses:

• Tales: A crowdsourced corpus based on reading of latin american short tales
• Aphasia: A crowdsourced corpus based in words categorized in 4 levels of difficulty
• Isolated words: A crowdsourced corpus based in isolated words

To download files from the Tales Project use

ops  \
    --output_folder tales/ \
    --output_file tales.txt  \
    --corpus tales

To download files from the Isolated Words Project use

ops  \
    --output_folder isolated_words/ \
    --output_file isolated_words.txt  \
    --corpus words

To download files from the Aphasia Project use

ops  \
    --output_folder aphasia/ \
    --output_file aphasia.txt  \
    --corpus aphasia

Parameters ussage

--download_all

You can download the whole corpus using the flag --download_all

ops  \
    --output_folder aphasia/ \
    --output_file aphasia.txt  \
    --corpus aphasia \
    --download_all

--from and --to

By default the page size is 500, to modify it use the args --from and --to i.e:

ops  \
    --from 500 \
    --to 1000 \
    --output_folder aphasia/ \
    --output_file aphasia.txt  \
    --corpus aphasia

If you use the flag --download_all with the flag --from the process will start in the specified arg from using a page size of 500

--extra_query_params

We also support an argument --extra_query_params which overwrites the --from, --to and --download_all arguments and downloads all files in the body response, you must define the --corpus argument anyway

ops  \
    --output_folder aphasia/ \
    --output_file aphasia_letters.txt  \
    --corpus aphasia \
    --extra_query_params "level_sentence__id__gte=846&level_sentence__id__lte=870"

Recursive Convert

The Open Speech Corpus stores its files in mp4 format, which is undesired for most audio processing tasks. To convert the files into a wav format, first install ffmpeg, then you can execute the recursive_convert utility which receives as first argument the source_folder with the mp4 files and as second argument the output folder i.e.:

recursive_convert aphasia aphasia_wav

CMU Sphinx Configuration

The Open Speech Corpus also defines some scripts to generate configurations for CMU Sphinx.

First initialize a project with the sphinx_train command

sphinxtrain -t simple_words setup

To generate a configuration use the command configure_sphinx, which creates the transcription, fileids, fillers and dic files.

configure_sphinx simple_words \
    --transcription_file words.txt \
    --etc_folder simple_words/etc \
    --test_size 0.5

Also you need to define a language model which receives the DB_NAME and the base project folder

generate_language_model simple_words simple_words

To delete the configuration files use the command clean_previous_configuration

clean_previous_configuration simple_words --etc_folder simple_words/etc/

HTK Configuration

The Open Speech Corpus also defines some scripts to train models using HTK

To generate a word grammar use

configure_htk \
    --transcription_file words.txt \
    --project_folder htk_words \
    --wav_folder words_wav \
    htk_words