oremi-ohunerin-listener 2.0.1

Oremi Ohunerin Listener bridges Oremi Ohunerin's real-time audio detection with MQTT, enabling seamless integration into home automation systems like Home Assistant

Oremi Ohunerin Listener

Oremi Ohunerin Listener is an implementation of Oremi Ohunerin, the real-time audio detection component of the Oremi Personal Assistant project. It connects to the Oremi Ohunerin WebSocket server, detects environmental sounds and wake words, and publishes the results to an MQTT broker.

This project is designed to integrate seamlessly with home automation systems like Home Assistant, IoT platforms, or any application that uses MQTT for communication. It is available as a Docker image for both linux/amd64 and linux/arm64 architectures.

Table of Contents

• Overview
  • What is Oremi Ohunerin?
  • What is Oremi Ohunerin Listener?
• Features
• Quick Start
  • Prerequisites
  • Docker Compose Example
  • Explanation
• Command-Line Arguments
  • General Usage
  • Listen Command
• MQTT Topics
• Customizing Detection with features.json
• Home Assistant Integration
  • Example Use Case
  • Home Assistant Configuration
    • configuration.yaml
  • Example Automations
    • 1. Snoring Detection
    • 2. Baby Cry Alert
    • 3. Cough Detection
    • 4. Wake Word Activation
  • Benefits of This Integration
• Contribute
• Versioning
• License

Overview

What is Oremi Ohunerin?

Oremi Ohunerin is the real-time audio detection component of the Oremi Personal Assistant project. It operates as a WebSocket server and is capable of identifying environmental sounds and specific wake words to activate Oremi. It leverages cutting-edge technologies such as:

• Tensorflow YAMNet: For comprehensive audio detection.
• PocketSphinx: For precise wake word identification.

It can recognize various sound categories, including but not limited to:

• Shouts
• Laughter
• Crying
• Snoring
• Coughing
• And more.

What is Oremi Ohunerin Listener?

Oremi Ohunerin Listener bridges the gap between Oremi Ohunerin and MQTT-based systems. It:

1. Connects to the Oremi Ohunerin WebSocket server.
2. Listens for detected sounds and wake words.
3. Publishes the results to an MQTT topic.

This allows you to integrate Oremi Ohunerin's audio detection capabilities into your MQTT-enabled ecosystem, such as Home Assistant, for advanced home automation and monitoring.

Features

• Real-Time Audio Detection: Detects environmental sounds and wake words in real time.
• Customizable Detection: Configure specific wake words and sounds to detect using a features.json file.
• MQTT Integration: Publishes detected sounds and wake words to an MQTT broker.
• Home Assistant Compatibility: Easily integrates with Home Assistant for automations, notifications, and dashboards.
• Customizable Threshold: Set a confidence score threshold for sound detection.
• Cooldown Period: Avoids publishing the same sound repeatedly within a specified delay.
• Docker Support: Available as a Docker image for easy deployment.
• Cross-Platform: Supports both linux/amd64 and linux/arm64 architectures.

Quick Start

Prerequisites

1. Docker: Ensure Docker is installed on your system.
2. MQTT Broker: An MQTT broker (e.g., Mosquitto) must be running and accessible.
3. Home Assistant: For integration and automation (optional but recommended).

Docker Compose Example

Here’s a quick example of how to use Oremi Ohunerin Listener with Docker Compose:

services:
  detector:
    image: demsking/oremi-ohunerin

  listener:
    image: demsking/oremi-ohunerin-listener
    devices:
      - /dev/snd:/dev/snd # Mount the microphone device
    volumes:
      - $XDG_RUNTIME_DIR/pulse/native:/var/config/pulse/native
      - ~/.config/pulse/cookie:/var/config/pulse/cookie
    environment:
      LOG_LEVEL: info
    command: listen --host detector --mqtt-broker mqtt.eclipseprojects.io --mqtt-port 1883

Explanation

• detector: The Oremi Ohunerin WebSocket server.
• listener: The Oremi Ohunerin Listener client.
  • Mounts the microphone device for audio input.
  • Connects to the MQTT broker at mqtt.eclipseprojects.io:1883.
  • Publishes detected sounds and wake words to the MQTT topic oremi/ohunerin.

Command-Line Arguments

General Usage

To see the general usage and available commands, run:

docker run -it --rm demsking/oremi-ohunerin-listener -h

Output:

usage: oremi-ohunerin-listener [-h] [-v] {listen} ...

Real-time ambient sound and wake word detection

positional arguments:
  {listen}
    listen       Start listening

options:
  -h, --help     show this help message and exit
  -v, --version  Show the version of the application.
  --list-devices List available input devices.

Listen Command

To see the options for the listen command, run:

docker run -it --rm demsking/oremi-ohunerin-listener listen -h

Output:

usage: oremi-ohunerin-listener listen [-h] [-l LANGUAGE] [-i DEVICE_INDEX] [-d DEVICE] [-t THRESHOLD] [--delay DELAY]
                                      [--features-file FEATURES_FILE] [--host HOST] [-p PORT] [--cert-file CERT_FILE] --mqtt-broker
                                      MQTT_BROKER [--mqtt-port MQTT_PORT] [--mqtt-username MQTT_USERNAME]
                                      [--mqtt-password MQTT_PASSWORD] [--mqtt-topic MQTT_TOPIC]

options:
  -h, --help            show this help message and exit
  -l LANGUAGE, --language LANGUAGE
                        The language to use for wake word detection and audio processing. Default is "fr" (French).
  -i DEVICE_INDEX, --device-index DEVICE_INDEX
                        Index of the audio device to be used for recording audio.
  -d DEVICE, --device DEVICE
                        Name of the device.
  -t THRESHOLD, --threshold THRESHOLD
                        The minimum confidence score required to process a sound event (default: 0.85).
  --delay DELAY         The cooldown period (in seconds) to wait before sending the same sound event again (default: 3).
  --features-file FEATURES_FILE
                        Path to the features.json file.
  --host HOST           Host address to connect to.
  -p PORT, --port PORT  Port number to connect to (default: 5023).
  --cert-file CERT_FILE
                        Path to the certificate file for secure connection.
  --mqtt-broker MQTT_BROKER
                        The address of the MQTT broker.
  --mqtt-port MQTT_PORT
                        The port of the MQTT broker. Default is 1883.
  --mqtt-username MQTT_USERNAME
                        The username for authenticating with the MQTT broker. Default is None.
  --mqtt-password MQTT_PASSWORD
                        The password for authenticating with the MQTT broker. Default is None.
  --mqtt-topic MQTT_TOPIC
                        The MQTT topic to publish audio detection events to. Default is "oremi/ohunerin".

MQTT Topics

Oremi Ohunerin Listener publishes detected sounds and wake words to the following MQTT topics:

• oremi/ohunerin/sound: For environmental sound detection events.
• oremi/ohunerin/wakeword: For wake word detection events.
• oremi/ohunerin/available: For availability status (online or offline).

Each message is published as a JSON object with the following structure:

{
  "type": "sound" | "wakeword",
  "sound": "detected_sound",
  "score": 0.85,
  "date": "2023-10-05T12:34:56Z"
}

Customizing Detection with features.json

The --features-file argument allows you to specify a JSON file that configures the behavior of the Oremi Ohunerin Listener. This file defines the features for wake word detection and sound detection, enabling you to customize which sounds and wake words the listener should detect.

Example features.json File

[
  {
    "name": "wakeword-detection",
    "language": "fr",
    "wakewords": [
      { "word": "alexa", "phones": ["aa ll ei kk ss aa"] },
      { "word": "ok google", "phones": ["oh k eh g uw g ah l"] }
    ],
    "discriminants": [
      { "word": "alex", "phones": ["aa ll ai kk ss"] },
      { "word": "google", "phones": ["g uw g ah l"] }
    ]
  },
  {
    "name": "sound-detection",
    "allowlist": [
      "Shout",
      "Bellows",
      "Children shouting",
      "Laughter",
      "Baby laughter",
      "Crying, sobbing",
      "Baby cry, infant cry",
      "Whistling",
      "Wheeze",
      "Snoring",
      "Cough",
      "Sneeze",
      "Burping",
      "Hiccup"
    ]
  }
]

Key Features

1. Wake Word Detection:

   • Defines the wake words and their phonetic representations (e.g., "alexa", "ok google").
   • Includes discriminants to reduce false positives (e.g., distinguishing between "alexa" and "alex").

2. Sound Detection:

   • Specifies an allowlist of sounds to detect (e.g., "Baby cry", "Snoring", "Cough").
   • If no allowlist is provided, the listener will detect all sounds by default.

Schema

The JSON file must adhere to the schema defined in the InitMessage.json schema file. This ensures compatibility with the Oremi Ohunerin WebSocket server.

Building a Phonetic Dictionary

To create custom phonetic representations for wake words, you can refer to the CMU Sphinx tutorial on building a phonetic dictionary. This guide provides step-by-step instructions for generating accurate phonetic spellings, which are essential for effective wake word detection.

Usage

To use the --features-file argument, provide the path to your JSON configuration file when starting the listener:

docker run -it --rm demsking/oremi-ohunerin-listener listen \
  --features-file /path/to/features.json \
  --mqtt-broker mqtt.eclipseprojects.io

Benefits

• Customization: Tailor the listener to detect specific wake words and sounds relevant to your use case.
• Efficiency: Reduce unnecessary processing by limiting detection to only the sounds and wake words you care about.
• Integration: Seamlessly integrate with home automation systems like Home Assistant by publishing only relevant events to MQTT.

By using the --features-file argument, you can configure the Oremi Ohunerin Listener to meet your specific needs, making it a powerful tool for real-time audio detection and automation.

Home Assistant Integration

Oremi Ohunerin Listener is an excellent tool for integrating real-time audio detection into Home Assistant. By publishing detected sounds and wake words to MQTT topics, you can create automations, notifications, and dashboards in Home Assistant based on audio events.

Example Use Case

Imagine you want to:

1. Detect when someone is snoring in a room.
2. Trigger a notification if a baby is crying.
3. Activate a wake word to start a specific automation.

With Oremi Ohunerin Listener and Home Assistant, you can achieve this seamlessly.

Home Assistant Configuration

Here’s an example of how to configure Home Assistant to work with Oremi Ohunerin MQTT:

configuration.yaml

mqtt:
  sensor:
    - name: "Wakeword"
      unique_id: "oremi_ohunerin_wakeword"
      state_topic: "oremi/ohunerin/wakeword"
      enabled_by_default: true
      icon: "mdi:waveform"
      device_class: "enum"
      options:
        - detected
        - 💤
      value_template: >-
        {% if value_json.sound == '...' %}
          💤
        {% else %}
          detected
        {% endif %}
      availability: &oremi-ohunerin-availability
        - topic: "oremi/ohunerin/available"
          payload_available: "online"
          payload_not_available: "offline"
      device: &oremi-ohunerin-device
        name: "Oremi Ohunerin"
        manufacturer: "Sébastien Demanou"
        configuration_url: "https://gitlab.com/demsking/oremi-ohunerin"
        identifiers: "oremi-ohunerin-1.0.0"
        sw_version: "1.0.0"

    - name: "Sound Detector"
      unique_id: "oremi_ohunerin_sound"
      state_topic: "oremi/ohunerin/sound"
      enabled_by_default: true
      icon: "mdi:waveform"
      device_class: "enum"
      options:
        - shout
        - bellows
        - children-shouting
        - laughter
        - baby-laughter
        - crying-sobbing
        - baby-cry-infant-cry
        - whistling
        - wheeze
        - snoring
        - cough
        - sneeze
        - burping
        - hiccup
        - 💤
      value_template: >-
        {% if value_json.sound == '...' %}
          💤
        {% else %}
          {{ value_json.sound }}
        {% endif %}
      availability: *oremi-ohunerin-availability
      device: *oremi-ohunerin-device

Example Automations

1. Snoring Detection

Send a notification when snoring is detected:

automation:
  - alias: "Notify on Snoring"
    description: Sends a notification with advice when snoring is detected.
    triggers:
      - trigger: state
        entity_id:
          - sensor.oremi_ohunerin_snoring_detector
    action:
      service: notify.notify
      data:
        title: "Snoring Detected"
        message: >
          Snoring detected! Consider elevating your head while sleeping,
          avoiding alcohol before bed, or trying nasal strips to improve airflow.
          Consult a healthcare professional if snoring persists.
    mode: single

2. Baby Cry Alert

Send an urgent notification when a baby cry is detected:

automation:
  - alias: Notify on Baby Cry
    description: Sends an urgent notification with soothing tips when a baby cry is detected.
    triggers:
      - trigger: state
        entity_id:
          - sensor.oremi_ohunerin_baby_cry_infant_cry_detector
    actions:
      - action: notify.notify
        data:
          title: "Baby Cry Detected"
          message: >
            Baby is crying! Check immediately. Try soothing techniques like
            gentle rocking, singing, or offering a pacifier. Ensure the baby
            is fed, dry, and comfortable.
    mode: single

3. Cough Detection

Send a notification with naturopathic advice when a cough is detected:

automation:
  - alias: Notify on Cough
    description: Sends a notification with naturopathic advice when a cough is detected.
    triggers:
      - trigger: state
        entity_id:
          - sensor.oremi_ohunerin_cough_detector
    actions:
      - action: notify.notify
        data:
          title: "Cough Detected"
          message: >
            Cough detected! Consider drinking warm water with honey and lemon,
            or try herbal teas like thyme or ginger to soothe your throat.
            Rest and stay hydrated!
    mode: single

4. Wake Word Activation

Trigger an automation when the wake word is detected:

automation:
  - alias: Wake Word Detected
    description: Triggered when the wake word is detected. Activates the Oremi Assistant script.
    triggers:
      - trigger: state
        entity_id:
          - sensor.oremi_ohunerin_wake_word
        to: detected
    actions:
      - target:
          entity_id: script.start_oremi_assistant
        action: script.turn_on
        data: {}
    mode: single

Benefits of This Integration

• Real-Time Alerts: Get instant notifications for specific sounds.
• Automations: Trigger actions based on detected sounds or wake words.
• Dashboards: Display sound detection states on your Home Assistant dashboard.
• Customizable: Easily adapt to your specific needs by modifying the configuration.

This integration makes Oremi Ohunerin Listener a powerful tool for enhancing your smart home with advanced audio detection capabilities.

Contribute

Please follow CONTRIBUTING.md.

Versioning

Given a version number MAJOR.MINOR.PATCH, increment the:

• MAJOR version when you make incompatible API changes,
• MINOR version when you add functionality in a backwards-compatible manner, and
• PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

See SemVer.org for more details.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at LICENSE.