ovos-media 0.0.1a19

ovos-core audio daemon client

OVOS media service

Media playback service for OpenVoiceOS

• Install
• Architecture
• Plugins
  • Media Plugins
  • OCP Plugins
• MPRIS integration
• Pipeline
  • ocp high
  • ocp medium
  • ocp fallback
• Favorite Songs
• Configuration

Install

pip install ovos-media to install this package and the default plugins.

In order to use ovos-media you need to enable the OCP pipeline in ovos-core and to disable the old audio service

disabling old OCP

{
  "enable_old_audioservice": false
}

Architecture

Plugins

WIP

Media Plugins

these plugins handle the actual track playback. OCP virtual player delegates media playback to these plugins

plugin	audio	video	web	remote	notes
ovos-media-plugin-simple	✔️	❌	❌	❌	default for audio
ovos-media-plugin-qt5	✔️	✔️	✔️	❌	WIP - recommended for embedded ovos-shell
ovos-media-plugin-mplayer	✔️	✔️	❌	❌	recommended for video
ovos-media-plugin-vlc	✔️	✔️	❌	❌
ovos-media-plugin-chromecast	✔️	✔️	❌	✔️	extra: cast_control for MPRIS interface
ovos-media-plugin-spotify	✔️	❌	❌	✔️	needs premium account extra: spotifyd for native spotify player
ovos-media-plugin-xdg	✔️	✔️	✔️	❌	xdg-open is for use inside a desktop session only
ovos-media-plugin-webbrowser	❌	❌	✔️	❌	webbrowser is for use inside a desktop session only

OCP Plugins

handle extracting playable streams and metadata, skills might require specific plugins and will be ignored if plugins are missing

these plugins are used when a sei// is requested explicitly by a skill, or when a url pattern matches

plugin	descripton	Stream Extractor Ids	url pattern
ovos-ocp-rss-plugin	rss feeds	rss//
ovos-ocp-bandcamp-plugin	bandcamp urls	bandcamp//	"bandcamp." in url
ovos-ocp-youtube-plugin	youtube urls	youtube// , ydl//, youtube.channel.live//	"youtube.com/" in url or "youtu.be/" in url
ovos-ocp-m3u-plugin	.pls and .m3u formats	m3u// , pls//	".pls" in uri or ".m3u" in uri
ovos-ocp-news-plugin	dedicated news websites	news//	any([uri.startswith(url) for url in URL_MAPPINGS])

MPRIS integration

Integration with MPRIS allows OCP to control external players

Pipeline

Enabling OCP pipeline

{
  // Intent Pipeline / plugins config
  "intents" : {
    // the pipeline is a ordered set of frameworks to send an utterance too
    // if one of the frameworks fails the next one is used, until an answer is found
    "pipeline": [
        "converse",
        "ocp_high",
        "...",
        "common_qa",
        "ocp_medium",
        "...",
        "ocp_fallback",
        "fallback_low"
    ]
  }
}

The dataset used to train the classifiers can be found here

Training code for classifiers used in the OCP pipeline can be found here

Details on the classifiers can be found here

ocp high

Before regular intent stage, taking into account current OCP state (media ready to play / playing)

Only matches if user unambiguously wants to trigger OCP

uses padacioso for exact matches

• play {query}
• previous (media needs to be loaded)
• next (media needs to be loaded)
• pause (media needs to be loaded)
• play / resume (media needs to be loaded)
• stop (media needs to be loaded)

from ocp_nlp.intents import OCPPipelineMatcher

ocp = OCPPipelineMatcher()
print(ocp.match_high("play metallica", "en-us"))
# IntentMatch(intent_service='OCP_intents',
#   intent_type='ocp:play',
#   intent_data={'media_type': <MediaType.MUSIC: 2>, 'query': 'metallica',
#                'entities': {'album_name': 'Metallica', 'artist_name': 'Metallica'},
#                'conf': 0.96, 'lang': 'en-us'},
#   skill_id='ovos.common_play', utterance='play metallica')

ocp medium

uses a binary classifier to detect if a query is about media playback

from ocp_nlp.intents import OCPPipelineMatcher

ocp = OCPPipelineMatcher()

print(ocp.match_high("put on some metallica", "en-us"))
# None

print(ocp.match_medium("put on some metallica", "en-us"))
# IntentMatch(intent_service='OCP_media',
#   intent_type='ocp:play',
#   intent_data={'media_type': <MediaType.MUSIC: 2>,
#                'entities': {'album_name': 'Metallica', 'artist_name': 'Metallica', 'movie_name': 'Some'},
#                'query': 'put on some metallica',
#                'conf': 0.9578441098114333},
#   skill_id='ovos.common_play', utterance='put on some metallica')

ocp fallback

Uses keyword matching and requires at least 1 keyword

OCP skills can provide these keywords at runtime, additional keywords for things such as media_genre were collected via SPARQL queries to wikidata

The list of bundled keywords can be found here

Skill names are automatically added as keywords, this ensures that if the skill name is present in an utterance the ocp_fallback pipeline will catch it

from ocp_nlp.intents import OCPPipelineMatcher

ocp = OCPPipelineMatcher()

print(ocp.match_medium("i wanna hear metallica", "en-us"))
# None

print(ocp.match_fallback("i wanna hear metallica", "en-us"))
#  IntentMatch(intent_service='OCP_fallback',
#    intent_type='ocp:play',
#    intent_data={'media_type': <MediaType.MUSIC: 2>,
#                 'entities': {'album_name': 'Metallica', 'artist_name': 'Metallica'},
#                 'query': 'i wanna hear metallica',
#                 'conf': 0.5027561091821287},
#    skill_id='ovos.common_play', utterance='i wanna hear metallica')

Favorite Songs

You can like a song that is currently playing via GUI and intent "I like that song"

Liked songs can be played via intent "play my favorite songs" or GUI

Configuration

under mycroft.conf

{
  // Configure ovos-media service
  // similarly to wakewords, configure any number of playback handlers
  // playback handlers might be local applications or even remote devices
  "media": {

    // enable MPRIS integration
    "enable_mpris": false,
    // have MPRIS control external players
    "manage_external_players": false,

    // order of preference to try playback handlers
    // if unavailable or unable to handle a uri, the next in list is used
    // NB: users may request specific handlers in the utterance

    // keys are the strings defined in "audio_players"
    "preferred_audio_services": ["gui", "vlc", "mplayer", "cli"],

    // keys are the strings defined in "web_players"
    "preferred_web_services": ["gui", "browser"],

    // keys are the strings defined in "video_players"
    "preferred_video_services": ["gui", "vlc"],

    // PlaybackType.AUDIO handlers
    "audio_players": {
        // vlc player uses a headless vlc instance to handle uris
        "vlc": {
            // the plugin name
            "module": "ovos-media-audio-plugin-vlc",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["VLC"],

            // deactivate a plugin by setting to false
            "active": true
        },
        // command line player uses configurable shell commands with file uris as arguments
        "cli": {
            // the plugin name
            "module": "ovos-media-audio-plugin-cli",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["Command Line"],

            // deactivate a plugin by setting to false
            "active": true
        },
        // gui uses mycroft-gui natively to handle uris
        "gui": {
            // the plugin name
            "module": "ovos-media-audio-plugin-gui",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["GUI", "Graphical User Interface"],

            // deactivate a plugin by setting to false
            "active": true
        }
    },

    // PlaybackType.VIDEO handlers
    "video_players": {
        // vlc player uses a headless vlc instance to handle uris
        "vlc": {
            // the plugin name
            "module": "ovos-media-video-plugin-vlc",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["VLC"],

            // deactivate a plugin by setting to false
            "active": true
        },
        // gui uses mycroft-gui natively to handle uris
        "gui": {
            // the plugin name
            "module": "ovos-media-video-plugin-gui",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["GUI", "Graphical User Interface"],

            // deactivate a plugin by setting to false
            "active": true
        }
    },

    // PlaybackType.WEBVIEW handlers
    "web_players": {
        // open url in the native browser
        "browser": {
            // the plugin name
            "module": "ovos-media-web-plugin-browser",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["Browser", "Local Browser", "Default Browser"],

            // deactivate a plugin by setting to false
            "active": true
        },
        // gui uses mycroft-gui natively to handle uris
        "gui": {
            // the plugin name
            "module": "ovos-media-web-plugin-gui",

            // friendly names a user may use to refer to this playback handler
            // those will be parsed by OCP and used to initiate
            // playback in the request playback handler
            "aliases": ["GUI", "Graphical User Interface"],

            // deactivate a plugin by setting to false
            "active": true
        }
    }
  }
}