Metadata-Version: 2.4
Name: ovoscope
Version: 1.6.2a2
Summary: End-to-end test framework for OpenVoiceOS skills
Author-email: JarbasAI <jarbasai@mailfence.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/TigreGotico/ovoscope
Project-URL: Documentation, https://github.com/TigreGotico/ovoscope/tree/master/docs
Project-URL: Issue Tracker, https://github.com/TigreGotico/ovoscope/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ovos-core>=2.0.4a2
Requires-Dist: ovos-utils>=0.12.0a1
Requires-Dist: pytest>=8
Requires-Dist: tomli>=2; python_version < "3.11"
Provides-Extra: pydantic
Requires-Dist: ovos-pydantic-models>=0.1.0; extra == "pydantic"
Provides-Extra: audio
Requires-Dist: ovos-audio>=1.3.0a1; extra == "audio"
Requires-Dist: ovos-spec-tools>=0.10.0a1; extra == "audio"
Provides-Extra: media
Requires-Dist: ovos-media>=0.0.2a3; extra == "media"
Provides-Extra: listener
Requires-Dist: ovos-dinkum-listener>=0.7.2a1; extra == "listener"
Provides-Extra: bench
Requires-Dist: numpy; extra == "bench"
Provides-Extra: tts
Requires-Dist: ovos-audio>=1.3.0a1; extra == "tts"
Requires-Dist: jiwer; extra == "tts"
Requires-Dist: ovos-utterance-normalizer; extra == "tts"
Requires-Dist: ovos-stt-plugin-fasterwhisper; extra == "tts"
Provides-Extra: dev
Requires-Dist: ovos-audio>=1.3.0a1; extra == "dev"
Requires-Dist: ovos-media>=0.0.2a3; extra == "dev"
Requires-Dist: ovos-dinkum-listener>=0.7.2a1; extra == "dev"
Requires-Dist: ovos-pydantic-models>=0.1.0; extra == "dev"
Requires-Dist: numpy; extra == "dev"
Requires-Dist: jiwer; extra == "dev"
Requires-Dist: ovos-utterance-normalizer; extra == "dev"
Requires-Dist: ovos-stt-plugin-fasterwhisper; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Dynamic: license-file

[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/TigreGotico/ovoscope)
[![PyPI](https://img.shields.io/pypi/v/ovoscope)](https://pypi.org/project/ovoscope/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/ovoscope/)
# OvoScope
**End-to-end testing for [OVOS](https://openvoiceos.org) skills.**
OvoScope runs a full OVOS Core pipeline in-process using a `FakeBus` — no server, no audio
stack, no network. Load real skill plugins, emit a test utterance, and assert on every bus
message that comes back: type, data, routing context, session state, and message ordering.
![image](https://github.com/user-attachments/assets/10a10ff5-64b7-42fd-86bd-cb6a5db769dd)
> Like a microscope for your OVOS skills.
---
## Features
| | |
|---|---|
| **Full pipeline** | Runs real intent pipeline plugins (Adapt, Padatious, Fallback, Converse, Common Query) |
| **Isolated** | Config isolation strips user preferences; deterministic `DEFAULT_TEST_PIPELINE` excludes AI/persona/OCP stages |
| **Ordered assertions** | Assert message type, data keys, routing context, and session state in sequence |
| **Recording mode** | Capture a live message sequence and save it as a JSON fixture — no manual construction needed |
| **Multi-turn** | Pass a list of utterances to test full conversational flows |
| **pytest fixture** | `minicroft` class-scoped fixture auto-discovered via the `pytest11` entry point |
| **Inject skills** | `extra_skills={id: SkillClass}` to load inline test skills without a PyPI entry point |
| **Inject messages** | `MiniCroft.inject_message()` to trigger non-utterance handlers (GUI events, timers, API calls) |
| **Typed models** | Optional `ovoscope[pydantic]` bridge to `ovos-pydantic-models` for schema-validated messages |
---
## Installation
```bash
pip install ovoscope
```
With optional typed message model support:
```bash
pip install ovoscope[pydantic]
```
---
## Quick Start
```python
import unittest
from ovos_bus_client.message import Message
from ovos_bus_client.session import Session
from ovoscope import End2EndTest
SKILL_ID = "ovos-skill-hello-world.openvoiceos"
session = Session("test-session")
utterance = Message(
    "recognizer_loop:utterance",
    {"utterances": ["hello world"], "lang": "en-US"},
    {"session": session.serialize(), "source": "A", "destination": "B"},
)
class TestHelloWorld(unittest.TestCase):
    def test_intent_match(self):
        End2EndTest(
            skill_ids=[SKILL_ID],
            source_message=utterance,
            expected_messages=[
                utterance,
                Message(f"{SKILL_ID}.activate", context={"skill_id": SKILL_ID}),
                Message(f"{SKILL_ID}:HelloWorldIntent",
                        data={"utterance": "hello world"}, context={"skill_id": SKILL_ID}),
                Message("mycroft.skill.handler.start", context={"skill_id": SKILL_ID}),
                Message("speak", data={"lang": "en-US"}, context={"skill_id": SKILL_ID}),
                Message("mycroft.skill.handler.complete", context={"skill_id": SKILL_ID}),
                Message("ovos.utterance.handled", context={"skill_id": SKILL_ID}),
            ],
        ).execute(timeout=10)
```
Only keys you specify in `expected.data` and `expected.context` are checked — extra keys in the
received message are ignored.
---
## Recording Mode
Don't know the exact message sequence yet? Record it from a live run:
```python
from ovoscope import End2EndTest
test = End2EndTest.from_message(
    message=utterance,
    skill_ids=[SKILL_ID],
    timeout=20,
)
test.save("tests/fixtures/hello_world.json")  # anonymises location data by default
```
Replay in CI:
```python
End2EndTest.from_path("tests/fixtures/hello_world.json").execute(timeout=10)
```
---
## pytest Fixture
The `minicroft` class-scoped fixture is auto-registered when ovoscope is installed.
No `setUp`/`tearDown` boilerplate needed:
```python
class TestMySkill:
    skill_ids = ["my-skill.author"]
    def test_something(self, minicroft):
        End2EndTest(
            minicroft=minicroft,
            skill_ids=self.skill_ids,
            source_message=utterance,
            expected_messages=[...],
        ).execute(timeout=10)
```
---
## Pipeline Control
OvoScope exposes composable pipeline stage lists so tests are deterministic regardless of which
AI plugins are installed on the host:
```python
from ovoscope import ADAPT_PIPELINE, PADATIOUS_PIPELINE, FALLBACK_PIPELINE, PERSONA_PIPELINE
# Adapt only — fastest
mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE)
# Full intent chain
mc = get_minicroft([SKILL_ID],
                   default_pipeline=ADAPT_PIPELINE + PADATIOUS_PIPELINE + FALLBACK_PIPELINE)
# Opt in to persona for AI testing
mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE)
```
`DEFAULT_TEST_PIPELINE` (the default when `isolate_config=True`) includes all standard built-in
stages and deliberately excludes persona, Ollama, OCP, and m2v plugins.
---
## Documentation
| Document | |
|---|---|
| [docs/usage-guide.md](docs/usage-guide.md) | **Start here** — 8 test patterns with full worked examples |
| [docs/ci-integration.md](docs/ci-integration.md) | Wiring ovoscope into GitHub Actions |
| [docs/minicroft.md](docs/minicroft.md) | `MiniCroft` and `get_minicroft()` reference |
| [docs/capture-session.md](docs/capture-session.md) | `CaptureSession` internals |
| [docs/end2end-test.md](docs/end2end-test.md) | `End2EndTest` full parameter reference |
| [docs/e2e-pipeline-harness.md](docs/e2e-pipeline-harness.md) | `E2EPipelineHarness` — testing a single pipeline plugin against raw bus messages |
| [docs/intent-cases.md](docs/intent-cases.md) | File-based intent test cases (`.intent.test`) via `register_intent_case_tests` |
| [docs/pydantic-integration.md](docs/pydantic-integration.md) | Typed message models with `ovos-pydantic-models` |
| [docs/cli.md](docs/cli.md) | `ovoscope` CLI — record/run/diff/validate/coverage/bus-coverage, plus `ovoscope-setup` |
| [FAQ.md](FAQ.md) | Common questions and gotchas |
---

---

## Credits

Developed by [TigreGótico](https://tigregotico.pt) for
[OpenVoiceOS](https://openvoiceos.org).

[![NGI0 Commons Fund](./ngi.png)](https://nlnet.nl/project/OpenVoiceOS)

This project was funded through the [NGI0 Commons Fund](https://nlnet.nl/commonsfund),
a fund established by [NLnet](https://nlnet.nl) with financial support from the
European Commission's [Next Generation Internet](https://ngi.eu) programme, under
the aegis of [DG Communications Networks, Content and Technology](https://commission.europa.eu/about-european-commission/departments-and-executive-agencies/communications-networks-content-and-technology_en)
under grant agreement No [101135429](https://cordis.europa.eu/project/id/101135429).

---

## License

[Apache 2.0](LICENSE)

---

## Contributing

PRs are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

---

## AI Disclosure

Parts of this project — including code, tests, and documentation — are developed with the
assistance of AI coding agents, under human review before merge. Commit messages and pull
request descriptions in the [git history](https://github.com/TigreGotico/ovoscope/commits/dev)
and [CHANGELOG.md](CHANGELOG.md) note when a change originated from an AI-assisted session, so
contributors and users can see where AI assistance has been applied.
