Development

During development, it may be useful to have a separate Docker image that is not named aukesch/speechmatching, which can then contain changes made during development. To support this, the speechmatching package will look for a Docker image named speechmatching before it looks for the Docker image aukesch/speechmatching. If the first docker image is available, the image under aukesch/ will not be used.

Local

Running locally requires (automatically pulled) the Docker image containing the acoustic binary. This image sets the environment variable ACOUSTIC_RUNNING_IN_DOCKER=1, which is not available when running outside the container.

When running speechmatching and an audio file needs to be processed, the package detects that it is not in Docker, then attempts to start a container from the aukesch/speechmatching Docker image (and attempt to pull it if it is not available locally). It will communicate with this container to process the audio file. When the program stops, the package tries to stop and remove the created container.

In other words, all that is needed is for Docker to be installed and set up correctly, and then the package can be run normally. These steps were explained in the previous section.

Docker

Alternatively, the speechmatching package entirely inside Docker. In this scenario, the package detects that the ACOUSTIC_RUNNING_IN_DOCKER=1 environment variable is present and attempts to interact with the acoustic binary locally within the same container.

To do this, include a Dockerfile in the directory of the code with a structure like:

FROM aukesch/speechmatching
COPY . .
# more code...
CMD ["python3", "main.py"]

After which the Docker image can be built and run.

Example

Most of the functions that are needed to run this on a basic level are in the speechmatching.recording submodule.

In the following example, we assume the user has several audio files:

./audio/speech1.mp3
./audio/speech2.3gp
./audio/speech3.wav
./audio/house1.mp3
./audio/house2.mp4
./audio/tree1.3gp
./audio/tree2.mp4
./audio/tree3.mp3
./audio/tree4.wav
./audio/unknown.mp3

The last file in this list is named unknown.mp3, and it is believed this audio file belongs to one of the three spoken words speech, house, or tree, for which we have several samples available.

A single recording can be loaded and analysed:

>>> from speechmatching.recording import Recording

>>> # load the recording
>>> speech1 = Recording('./audio/speech1.mp3')

>>> # get the transcript
>>> transcript = speech1.transcript

>>> # print the most likely text
>>> print(transcript.text)

>>> # print the top likely texts
>>> print(transcript.probable_texts())

>>> # and calculate similarity scores to other recordings
>>> speech2 = Recording('./audio/speech2.3gp')
>>> print(speech2.similarity(speech1))
>>> # same as
>>> print(speech2.transcript.similarity(speech1.transcript))

These samples are sorted into groups:

>>> from speechmatching.recording import Group, Recording

>>> groups = [
...     Group(
...         identifier='speech',
...         recordings=[
...             Recording('./audio/speech1.mp3'),
...             Recording('./audio/speech2.3gp'),
...             Recording('./audio/speech3.wav')
...         ]
...     ),
...     Group(
...         identifier='house',
...         recordings=[
...             Recording('./audio/house1.mp3'),
...             Recording('./audio/house2.mp4')
...         ]
...     ),
...     Group(
...         identifier='tree',
...         recordings=[
...             Recording('./audio/tree1.3gp'),
...             Recording('./audio/tree2.mp4'),
...             Recording('./audio/tree3.mp3'),
...             Recording('./audio/tree4.wav')
...         ]
...     )
... ]

>>> # load the unknown recording and match it
>>> unknown = Recording('./audio/unknown.mp3')
>>> match = unknown.match(groups)
>>> print(match.identifier)
>>> # best matching group is printed here

There are more possibilities, with much greater control over the process by using various arguments available to the functions around normalization of strings, calculating similarity scores, and strategies for finding the best matching group.

Using the function .match(...) of the Recording instance uses the combination of normalization and matching algorithms that was found to work best in the thesis for which this software was written.

More examples can be found in the examples/ directory.