This release is a pre-release and may not be stable for production use.
OvoScope
End-to-end testing for OVOS skills.
OvoScope runs a full OVOS Core pipeline in-process using a FakeBus — no server, no audio
stack, no network. Load real skill plugins, emit a test utterance, and assert on every bus
message that comes back: type, data, routing context, session state, and message ordering.
Like a microscope for your OVOS skills.
Features
| Full pipeline | Runs real intent pipeline plugins (Adapt, Padatious, Fallback, Converse, Common Query) |
| Isolated | Config isolation strips user preferences; deterministic DEFAULT_TEST_PIPELINE excludes AI/persona/OCP stages |
| Ordered assertions | Assert message type, data keys, routing context, and session state in sequence |
| Recording mode | Capture a live message sequence and save it as a JSON fixture — no manual construction needed |
| Multi-turn | Pass a list of utterances to test full conversational flows |
| pytest fixture | minicroft class-scoped fixture auto-discovered via the pytest11 entry point |
| Inject skills | extra_skills={id: SkillClass} to load inline test skills without a PyPI entry point |
| Inject messages | MiniCroft.inject_message() to trigger non-utterance handlers (GUI events, timers, API calls) |
| Typed models | Optional ovoscope[pydantic] bridge to ovos-pydantic-models for schema-validated messages |
Installation
pip install ovoscope
With optional typed message model support:
pip install ovoscope[pydantic]
Quick Start
import unittest
from ovos_bus_client.message import Message
from ovos_bus_client.session import Session
from ovoscope import End2EndTest
SKILL_ID = "ovos-skill-hello-world.openvoiceos"
session = Session("test-session")
utterance = Message(
"recognizer_loop:utterance",
{"utterances": ["hello world"], "lang": "en-US"},
{"session": session.serialize(), "source": "A", "destination": "B"},
)
class TestHelloWorld(unittest.TestCase):
def test_intent_match(self):
End2EndTest(
skill_ids=[SKILL_ID],
source_message=utterance,
expected_messages=[
utterance,
Message(f"{SKILL_ID}.activate", context={"skill_id": SKILL_ID}),
Message(f"{SKILL_ID}:HelloWorldIntent",
data={"utterance": "hello world"}, context={"skill_id": SKILL_ID}),
Message("mycroft.skill.handler.start", context={"skill_id": SKILL_ID}),
Message("speak", data={"lang": "en-US"}, context={"skill_id": SKILL_ID}),
Message("mycroft.skill.handler.complete", context={"skill_id": SKILL_ID}),
Message("ovos.utterance.handled", context={"skill_id": SKILL_ID}),
],
).execute(timeout=10)
Only keys you specify in expected.data and expected.context are checked — extra keys in the
received message are ignored.
Recording Mode
Don't know the exact message sequence yet? Record it from a live run:
from ovoscope import End2EndTest
test = End2EndTest.from_message(
message=utterance,
skill_ids=[SKILL_ID],
timeout=20,
)
test.save("tests/fixtures/hello_world.json") # anonymises location data by default
Replay in CI:
End2EndTest.from_path("tests/fixtures/hello_world.json").execute(timeout=10)
pytest Fixture
The minicroft class-scoped fixture is auto-registered when ovoscope is installed.
No setUp/tearDown boilerplate needed:
class TestMySkill:
skill_ids = ["my-skill.author"]
def test_something(self, minicroft):
End2EndTest(
minicroft=minicroft,
skill_ids=self.skill_ids,
source_message=utterance,
expected_messages=[...],
).execute(timeout=10)
Pipeline Control
OvoScope exposes composable pipeline stage lists so tests are deterministic regardless of which AI plugins are installed on the host:
from ovoscope import ADAPT_PIPELINE, PADATIOUS_PIPELINE, FALLBACK_PIPELINE, PERSONA_PIPELINE
# Adapt only — fastest
mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE)
# Full intent chain
mc = get_minicroft([SKILL_ID],
default_pipeline=ADAPT_PIPELINE + PADATIOUS_PIPELINE + FALLBACK_PIPELINE)
# Opt in to persona for AI testing
mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE)
DEFAULT_TEST_PIPELINE (the default when isolate_config=True) includes all standard built-in
stages and deliberately excludes persona, Ollama, OCP, and m2v plugins.
Documentation
| Document | |
|---|---|
| docs/usage-guide.md | Start here — 8 test patterns with full worked examples |
| docs/ci-integration.md | Wiring ovoscope into GitHub Actions |
| docs/minicroft.md | MiniCroft and get_minicroft() reference |
| docs/capture-session.md | CaptureSession internals |
| docs/end2end-test.md | End2EndTest full parameter reference |
| docs/e2e-pipeline-harness.md | E2EPipelineHarness — testing a single pipeline plugin against raw bus messages |
| docs/intent-cases.md | File-based intent test cases (.intent.test) via register_intent_case_tests |
| docs/pydantic-integration.md | Typed message models with ovos-pydantic-models |
| docs/cli.md | ovoscope CLI — record/run/diff/validate/coverage/bus-coverage, plus ovoscope-setup |
| FAQ.md | Common questions and gotchas |
Credits
Developed by TigreGótico for OpenVoiceOS.
This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101135429.
License
Contributing
PRs are welcome! See CONTRIBUTING.md for guidelines.
AI Disclosure
Parts of this project — including code, tests, and documentation — are developed with the assistance of AI coding agents, under human review before merge. Commit messages and pull request descriptions in the git history and CHANGELOG.md note when a change originated from an AI-assisted session, so contributors and users can see where AI assistance has been applied.
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ovoscope-1.6.10a1.tar.gz.
File metadata
- Download URL: ovoscope-1.6.10a1.tar.gz
- Upload date:
- Size: 146.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
cfd4a8f2fbcfa924412c0c63be9ae2f1098c1e4bb724ad5787a85a3e44e2bf26
|
|
| MD5 |
aecf810ecd271c43801d42e74844dbf0
|
|
| BLAKE2b-256 |
5ee8e584d86690bc04131e2481e767a909a3ab5244518e9403483d233475c819
|
File details
Details for the file ovoscope-1.6.10a1-py3-none-any.whl.
File metadata
- Download URL: ovoscope-1.6.10a1-py3-none-any.whl
- Upload date:
- Size: 159.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
cd305e49d63d3c7e0a38ebeb2b9b4aae1d9824ef7fac861a63308388e436dbdb
|
|
| MD5 |
20ecb1cf9e62d5a58ebfabbfa9c4f623
|
|
| BLAKE2b-256 |
d3515cd65eabe9484609fae8ba7d9f23e297c623a0bd4d2ca577f8fbcbd758df
|
