Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Ask DeepWiki PyPI License Python

OvoScope

End-to-end testing for OVOS skills.

OvoScope runs a full OVOS Core pipeline in-process with a FakeBus. It needs no server, no audio stack, and no network. Load real skill plugins, send a test utterance, and check every bus message that comes back: type, data, routing context, session state, and message order.

image


Features

Full pipeline Runs real intent pipeline plugins (Adapt, Padatious, Fallback, Converse, Common Query)
Isolated Config isolation strips user preferences, and the deterministic DEFAULT_TEST_PIPELINE excludes AI, persona, and OCP stages
Ordered assertions Checks message type, data keys, routing context, and session state in order
Recording mode Captures a live message sequence and saves it as a JSON fixture. No manual construction needed
Multi-turn Pass a list of utterances to test full conversational flows
pytest fixture The minicroft class-scoped fixture is auto-discovered through the pytest11 entry point
Inject skills Use extra_skills={id: SkillClass} to load inline test skills without a PyPI entry point
Inject messages Use MiniCroft.inject_message() to trigger non-utterance handlers (GUI events, timers, API calls)
Typed models The optional ovoscope[pydantic] bridge adds schema-validated messages through ovos-pydantic-models

Installation

pip install ovoscope

To add typed message model support:

pip install ovoscope[pydantic]

Quick Start

import unittest
from ovos_bus_client.message import Message
from ovos_bus_client.session import Session
from ovoscope import End2EndTest
SKILL_ID = "ovos-skill-hello-world.openvoiceos"
session = Session("test-session")
utterance = Message(
    "recognizer_loop:utterance",
    {"utterances": ["hello world"], "lang": "en-US"},
    {"session": session.serialize(), "source": "A", "destination": "B"},
)
class TestHelloWorld(unittest.TestCase):
    def test_intent_match(self):
        End2EndTest(
            skill_ids=[SKILL_ID],
            source_message=utterance,
            expected_messages=[
                utterance,
                Message(f"{SKILL_ID}.activate", context={"skill_id": SKILL_ID}),
                Message(f"{SKILL_ID}:HelloWorldIntent",
                        data={"utterance": "hello world"}, context={"skill_id": SKILL_ID}),
                Message("mycroft.skill.handler.start", context={"skill_id": SKILL_ID}),
                Message("speak", data={"lang": "en-US"}, context={"skill_id": SKILL_ID}),
                Message("mycroft.skill.handler.complete", context={"skill_id": SKILL_ID}),
                Message("ovos.utterance.handled", context={"skill_id": SKILL_ID}),
            ],
        ).execute(timeout=10)

OvoScope checks only the keys you list in expected.data and expected.context. It ignores extra keys in the received message.

Recording Mode

If you do not know the exact message sequence yet, record it from a live run:

from ovoscope import End2EndTest
test = End2EndTest.from_message(
    message=utterance,
    skill_ids=[SKILL_ID],
    timeout=20,
)
test.save("tests/fixtures/hello_world.json")  # anonymizes location data by default

Replay the fixture in CI:

End2EndTest.from_path("tests/fixtures/hello_world.json").execute(timeout=10)

pytest Fixture

OvoScope auto-registers the minicroft class-scoped fixture on install. You do not need setUp/tearDown boilerplate:

class TestMySkill:
    skill_ids = ["my-skill.author"]
    def test_something(self, minicroft):
        End2EndTest(
            minicroft=minicroft,
            skill_ids=self.skill_ids,
            source_message=utterance,
            expected_messages=[...],
        ).execute(timeout=10)

Pipeline Control

OvoScope exposes composable pipeline stage lists so tests stay deterministic regardless of which AI plugins are installed on the host:

from ovoscope import ADAPT_PIPELINE, PADATIOUS_PIPELINE, FALLBACK_PIPELINE, PERSONA_PIPELINE
# Adapt only: fastest
mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE)
# Full intent chain
mc = get_minicroft([SKILL_ID],
                   default_pipeline=ADAPT_PIPELINE + PADATIOUS_PIPELINE + FALLBACK_PIPELINE)
# Opt in to persona for AI testing
mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE)

DEFAULT_TEST_PIPELINE is the default when isolate_config=True. It includes all standard built-in stages and leaves out persona, Ollama, OCP, and m2v plugins.

Documentation

Document
docs/usage-guide.md Start here: 8 test patterns with full worked examples
docs/ci-integration.md Wiring OvoScope into GitHub Actions
docs/minicroft.md MiniCroft and get_minicroft() reference
docs/capture-session.md CaptureSession internals
docs/end2end-test.md End2EndTest full parameter reference
docs/e2e-pipeline-harness.md E2EPipelineHarness — testing a single pipeline plugin against raw bus messages
docs/intent-cases.md File-based intent test cases (.intent.test) via register_intent_case_tests
docs/pydantic-integration.md Typed message models with ovos-pydantic-models
docs/cli.md ovoscope CLI — record/run/diff/validate/coverage/bus-coverage, plus ovoscope-setup
FAQ.md Common questions and gotchas


OvoScope is part of the OpenVoiceOS tooling suite:

  • ovos-core: the OVOS assistant core that OvoScope tests skills against.
  • ovos-workshop: the skill base classes that OvoScope loads and drives.
  • ovos-bus-client: the message bus client behind FakeBus and Message.
  • ovos-test-harness: a companion test harness for OVOS components.

Credits

Developed by TigreGótico for OpenVoiceOS.

NGI0 Commons Fund

This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101135429.


License

Apache 2.0


Contributing

PRs are welcome. See CONTRIBUTING.md for guidelines.


AI Disclosure

Parts of this project — including code, tests, and documentation — are developed with the assistance of AI coding agents, under human review before merge. Commit messages and pull request descriptions in the git history and CHANGELOG.md note when a change originated from an AI-assisted session, so contributors and users can see where AI assistance has been applied.

Metadata

Release files for ovoscope 1.12.1a1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ovoscope 1.12.1a1
File Size Uploaded Provenance Actions
ovoscope-1.12.1a1.tar.gz 190.3 kB Without provenance Details for ovoscope-1.12.1a1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ovoscope 1.12.1a1
File Interpreter ABI Platform Actions
ovoscope-1.12.1a1-py3-none-any.whl Python 3 none any Details for ovoscope-1.12.1a1-py3-none-any.whl

Total release size: 393.7 kB

Release files / ovoscope-1.12.1a1.tar.gz

Download URL ovoscope-1.12.1a1.tar.gz
Size 190.3 kB
Tags Source
SHA-256 checksum
How to use checksums
eee454ecf274b5ecafd1730bdad280d1484dfbb8bfbcb8d45f148a1645dbd900
BLAKE2b-256 checksum
How to use checksums
e9f193bdba6d00db64e9e1c475bb34635a0f403eaa4ac6198cfe3b3a44429969
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / ovoscope-1.12.1a1-py3-none-any.whl

Download URL ovoscope-1.12.1a1-py3-none-any.whl
Size 203.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8335f91fb941dd763422472808f16c64c29533ad4fcba6158f02fd8b63f8c8e8
BLAKE2b-256 checksum
How to use checksums
4d1a8670788cb839e4458bdad190542a7fd6788d2a2422713e46a4736eb70909
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for ovoscope 1.12.1a1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page