Metadata-Version: 2.4
Name: pytest-qamule
Version: 0.1.0
Summary: A pytest plugin for QAMule automation testing with device fixtures, live pause checkpoints, and AI/human-friendly reports.
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: pytest>=9.1.1
Requires-Dist: uiautomator2>=3.6.0
Description-Content-Type: text/markdown

# pytest-qamule

A pytest plugin for QAMule automation testing, integrating device control, live pause debugging/checkpoint, and AI/human-friendly test reports.

## What it does

- **Device fixtures**: expose `uiautomator2` devices as pytest fixtures such as `d`, `phone`, or `tablet`.
- **Live pause**: pause a failing test, inspect the app state, then resume or abort from the CLI.
- **Checkpoints**: add explicit human/AI review points inside tests with the `checkpoint` fixture.
- **Live reports**: write an event-backed report while pytest runs, then inspect it from a browser or JSON CLI.

## Install

```bash
uv add pytest-qamule
```

The package exposes a pytest plugin through the `qamule` pytest11 entry point and a `qamule` CLI script.

## Device fixtures

`pytest-qamule` exposes uiautomator2 devices as pytest fixtures. With no options, a default `d` fixture connects through `u2.connect()`:

```bash
pytest
```

```python
def test_home_screen(d):
    d.app_start("com.example.app")
```

Register named devices with repeated `--device NAME:SERIAL` options:

```bash
pytest --device phone:emulator-5554 --device tablet:0123456789
```

```python
def test_home_screen(phone):
    phone.app_start("com.example.app")
```

## Pause Sessions

QAMule creates a session directory directly under the current working directory. Pause protocol files are written under `<session-id>/pauses/<pause-id>/`. Pytest prints the session id at startup:

```text
QAMule Session ID: s-20260624-153000-a1b2c3
QAMule Session Dir: /path/to/project/s-20260624-153000-a1b2c3
```

The session is marked finished during pytest unconfigure. Generated `s-*/` session directories are ignored by git.

## Failure Pause

Enable live pause on test failures with `--qamule-pause-on-failure`, or use the shorter `--pause-on-failure` alias for local debugging.

```bash
pytest --qamule-pause-on-failure -s
pytest --pause-on-failure -s
```

Failure pause triggers only when the pytest call phase fails and the report is not an expected xfail. It pauses after the failing call report is produced, then resumes teardown and the rest of the run after a `resume` command.

Attach preset images before the failure with the auto-loaded `qamule_pause` fixture:

```python
def test_failure(qamule_pause):
    qamule_pause.add_image("before-failure.png")
    qamule_pause.add_images(["detail-1.png", "detail-2.png"])
    assert False, "boom"
```

After resume or abort, failure pause data is attached to the pytest call report for future reporting integrations:

```python
def pytest_runtest_logreport(report):
    if report.when == "call" and hasattr(report, "qamule_pause"):
        message = report.qamule_pause["message"]
        images = report.qamule_pause["images"]
```

The same payload is also added to `report.user_properties` with the key `qamule_pause`.

## Checkpoints

The `checkpoint` fixture is auto-loaded by the pytest plugin and can be used directly in tests. It pauses immediately when called, without requiring `--pause-on-failure`.

```python
def test_checkpoint(checkpoint):
    response = checkpoint("verify app state", images=["checkpoint.png"])
    assert response.result is True
    assert response.message == "checkpoint accepted"
    assert response.images[0].base64
```

`checkpoint(...)` returns a `CheckpointResult` with:

```python
response.result
response.message
response.images
```

Resume a checkpoint with a result:

```bash
qamule pause resume p-000001 --session-id <session-id> --result true --message "checkpoint accepted"
```

For non-interactive runs, mock checkpoint resume results at pytest startup:

```bash
pytest --qamule-checkpoint-mock-result true -s
pytest --qamule-checkpoint-mock-result false -s
pytest --qamule-checkpoint-mock-result none -s
pytest --qamule-checkpoint-mock-result true --qamule-checkpoint-mock-message "approved by reviewer" -s
```

Mocked checkpoints still write pause and result protocol files, then immediately resume. By default, the checkpoint message is the selected mock value: `true`, `false`, or `none`. Use `--qamule-checkpoint-mock-message` to attach a more descriptive review message.

`--result` must be `true`, `false`, or `none`. Checkpoint pauses do not support `abort`. Checkpoint timeouts automatically resume with `response.result is None` and `response.message == "pause timed out"`; they do not automatically fail the test.

## CLI

`qamule pause` commands emit JSON and require `--session-id`:

```bash
qamule pause status --session-id <session-id>
qamule pause ls --session-id <session-id>
qamule pause show <pause-id> --session-id <session-id>
qamule pause resume <pause-id> --session-id <session-id> --message "checked" --image screenshot.png
qamule pause abort <pause-id> --session-id <session-id> --message "stop run" --image screenshot.png
qamule pause watch --session-id <session-id> --timeout 30 --interval 0.5
```

Command behavior:

- `status`: prints whether the session has active pauses; exits `0` when active pauses exist and `2` otherwise.
- `ls`: lists all pauses for the session.
- `show`: prints one pause by id; exits `2` if the pause is missing.
- `resume`: writes a resume command for an active pause.
- `abort`: writes an abort command for an active failure pause; checkpoint abort exits `5` and does not write a command.
- `watch`: waits for an active pause, a finished session, or timeout; timeout exits `2`.

`resume` continues a pause. `abort` stops a failure pause by raising `KeyboardInterrupt` in pytest.

## Images

The `--image` option accepts a file path and can be repeated. File contents are encoded as base64, and the file name is stored as the image alias. Duplicate image content is removed while preserving first-seen order. The same alias can appear more than once when the underlying base64 content is different.

`qamule` commands keep terminal JSON concise: attached images are reported by alias or count, and `base64` fields are omitted from command output. Full base64 image data is written only to the pause protocol files for the report UI and protocol readers.

Images can come from code-side presets and CLI commands. They are merged in this order:

1. Code-side preset images from `qamule_pause` or `checkpoint(..., images=[...])`
2. CLI images from `qamule pause resume ... --image ...` or `qamule pause abort ... --image ...`

Each image has this shape in protocol files, checkpoint responses, and failure report attachments:

```python
{
    "alias": "image",
    "base64": "...",
}
```

Example failure workflow:

```bash
pytest test_a.py --pause-on-failure -s
qamule pause watch --session-id <session-id>
qamule pause show p-000001 --session-id <session-id>
qamule pause resume p-000001 --session-id <session-id> --message "checked failure state" --image screenshot.png
```

Example checkpoint workflow:

```bash
pytest test_a.py -s
qamule pause watch --session-id <session-id>
qamule pause resume p-000001 --session-id <session-id> --result true --message "checkpoint accepted" --image final.png
```

## Live Reports

Enable the live report store with `--qamule-report`:

```bash
pytest --qamule-report
pytest --qamule-report --pause-on-failure -s
```

Report files are written directly in the QAMule session directory while pytest is still running:

```text
<session-id>/
events.jsonl
state.json
```

`events.jsonl` is the append-only event stream, and `state.json` records whether the run is running, finished, aborted, interrupted, or lost. QAMule does not store a duplicated report snapshot; report views are built dynamically from `<session-id>/events.jsonl` plus pause protocol files under `<session-id>/pauses/`. Evidence images are stored once in pause protocol JSON as `alias` and `base64`, and report views reuse that data instead of creating image asset files or copying base64 into report events. If the pytest process disappears while `state.json` still says running, report readers display the session as lost instead of treating the report as invalid.

Serve the human-friendly live report UI with:

```bash
qamule report serve --session-id <session-id>
qamule report serve --session-id <session-id> --host 127.0.0.1 --port 8765
qamule report serve --session-id <session-id> -q
```

The report server opens the browser UI by default and exposes JSON endpoints backed by the same dynamically built report view. Pass `-q` or `--quite` to print the URL without opening a browser. The browser uses server-sent events to render new results as they arrive. The UI logo is served from packaged report assets, so it does not require a separate `logo.jpg` in the current working directory.

All `qamule report` commands read `events.jsonl` and `state.json` from `<session-id>/` under the current working directory. To inspect a copied report, run the command from the directory that contains the session directory.

AI agents and scripts can query focused slices of the report without reading the full file:

```bash
qamule report status --session-id <session-id>
qamule report list --session-id <session-id>
qamule report list --session-id <session-id> --outcome failed
qamule report list --session-id <session-id> --with evidence
qamule report show tests/test_a.py::test_example --session-id <session-id>
qamule report failures --session-id <session-id>
qamule report checkpoints --session-id <session-id>
qamule report rebuild --session-id <session-id>
```

`qamule report list` can filter by `--outcome passed|failed|skipped|running|unknown` and by `--with checkpoint|failure-evidence|evidence`.

The default report view focuses on test outcomes, checkpoint decisions, failure evidence, messages, results, and images. Detailed event history remains available through `<session-id>/events.jsonl`, while pause details and images remain in `<session-id>/pauses/`.

When a pause has both code-side preset images and CLI-attached images, the report keeps the legacy merged `images` list for compatibility and also exposes `preset_images` and `attached_images` so the browser can display them in separate groups.

To avoid duplicate image storage, `pause.json` owns code-side preset image base64 and `command.json` owns CLI-attached image base64. `result.json` records only the pause decision fields and does not store image base64.

## Development

Pause and report CLI commands share the same public JSON output helper. When changing CLI output, keep terminal JSON free of `base64` fields and cover the shared helper plus the affected feature tests.

```bash
uv sync
uv run pytest
uv run pytest tests/unit/test_output.py
uv run pytest tests/unit/features/device tests/integration/features/device
uv run pytest tests/unit/features/pause tests/integration/features/pause
uv run pytest tests/unit/features/report tests/integration/features/report
```
