Metadata-Version: 2.5
Name: qa-agent-kit
Version: 0.6.0
Summary: The QA agent kit: mutation testing for any repo, prose gates with a claims ledger, and real-device QA for Apple apps, with the procedures an AI coding agent follows.
Project-URL: Homepage, https://github.com/younatics/QAKit
Project-URL: Changelog, https://github.com/younatics/QAKit/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/younatics/QAKit/blob/main/docs/getting-started.md
Author: younatics
License-Expression: MIT
License-File: LICENSE
Keywords: agents,claude-code,codex,ios,mutation-testing,prose,qa,xcuitest
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Provides-Extra: test
Requires-Dist: pyflakes; extra == 'test'
Requires-Dist: pytest; extra == 'test'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/banner.svg" alt="QAKit: the QA agent kit" width="100%">
</p>

# QAKit

**The QA agent kit.** Procedures an AI coding agent follows to test like a QA engineer, the tools that run the checks,
and the evidence format a release verdict cites. Three layers, one repo:

| Layer | For | Skill | Tools |
|---|---|---|---|
| Code | any repo with tests | `skills/qa-rigor` | `tools/mutate.py` (mutation testing, Python and C-family syntax) |
| Real device | Apple apps | `skills/device-qa` | `tools/device_qa.py` (test, traits, perf, upgrade, release, one lock per phone), `tools/release_checks.py`, `recording_frames.swift`, `screenshot_contrast.swift`, Swift package `QAKit` for XCUITest |
| Prose | memos, READMEs, papers | `skills/prose-qa` | `tools/prose_check.py` (gates and a claims ledger) |

QAKit grew out of shipping several App Store apps as a solo developer whose QA team is an agent. Every check in it was
added after a defect that green tests let through: a `>=` that became `>` at every grading boundary with 13 mutants
surviving, a review note over the 4,000-character limit, a clipped caption nobody looked at, a What's New that still
announced the first release in version 1.1.

## See it

<p align="center"><img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/demo-walk.gif" alt="An agent walking an app on a real iPhone: onboarding, the home screen at every scroll position, each button tapped once" width="300"></p>

`QA.walk` on a real iPhone: the onboarding pages, the home screen at every scroll position, every visible button tapped once
and screenshotted, the accessibility audit per screen. These 29 frames are one walkthrough test's attachments, assembled by
`tools/shots_gif.py` from the result bundle (the app is the author's, live on the App Store).

<p align="center"><img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/demo-variants.gif" alt="The Notes example app's list, settings and search screens in the default, dark and large-text variants, one phone, one command" width="300"></p>

The same walk in three variants: `device_qa.py test --variant default --variant dark --variant large-text` switches the
phone's appearance and text size between runs. These nine frames are the `examples/Notes` app's list, settings and
search screens from one such run.

## What makes it different

- **Systematic, not random.** `QA.sweepButtons` taps every visible button on every screen at every scroll position,
  screenshots what opened, checks the app is still in front and goes back. Monkey testers tap at random; agents reviewing a
  contact sheet need every control seen once.
- **The whole phone changes, not the app.** `device_qa.py test --variant dark` switches the device to dark mode, large
  accessibility text, a simulated location or VoiceOver through Xcode 27's `devicectl`, runs the suite, and restores iOS
  defaults. System alerts and other apps render the same way the user will see them.
- **Apple's accessibility audit on every screen,** collected into one report per screen instead of stopping at the first
  issue, with an allow list for reviewed exceptions.
- **Test strength is measured.** Coverage says a line ran. Mutation testing says whether a test would notice if the
  line were wrong. The gate is "every mutant on a changed line is dead or explained".
- **Prose gets the same treatment.** Deterministic gates, then a claims ledger where every number is traced to a quote,
  then a cold read by a reader who did not write it.
- **Written for agents.** The skills are `SKILL.md` files a coding agent loads. The protocol has 180 checks with IDs,
  pass criteria, methods and sources, so an agent can cite what it did and what it skipped.

## Quick start

```bash
pip install qa-agent-kit                           # qakit-mutate, qakit-prose, qakit-device, qakit-release-checks, qakit-install-skills
# or from source
git clone https://github.com/younatics/QAKit.git
cd QAKit && python3 -m pytest -q tests          # 150 tests, no dependency beyond pytest

# mutation testing on any project
python3 tools/mutate.py --root ../myproject --source 'src/**/*.py' --test '{orig}/.venv/bin/python -m pytest -x -q'

# prose
python3 tools/prose_check.py ../memo.md --claims claims.csv

# an Apple app on a USB-attached test iPhone (macOS, Xcode 27)
python3 tools/device_qa.py doctor --manifest ../myapp/release/testing/device_qa.json
python3 tools/device_qa.py test --manifest ../myapp/release/testing/device_qa.json --suite qa --variant dark
```

Give the procedures to your agent, whichever one you use:

```bash
claude plugin marketplace add younatics/QAKit && claude plugin install qakit@younatics   # Claude Code: skills + MCP tools
python3 tools/install_skills.py --target claude --scope project      # or codex, cursor, gemini, all; --scope user
claude mcp add qakit -- python3 /path/to/qakit/mcp/server.py        # any MCP client: doctor, test, shots, crashes, release checks, mutation, prose, verdict
```

Then read [docs/getting-started.md](docs/getting-started.md). [AGENTS.md](AGENTS.md) tells any agent which skill to load
for which job.

## Documentation

| Document | What it covers |
|---|---|
| [docs/getting-started.md](docs/getting-started.md) | Installing, linking the Swift package, the first walkthrough test, the first device run |
| [docs/device-qa-protocol.md](docs/device-qa-protocol.md) | The protocol: 24 areas, 180 checks with IDs, exit criteria, what only a human can do ([한국어판](docs/ko/device-qa-protocol.md)) |
| [docs/manifest.md](docs/manifest.md) | Every key of the `device_qa.json` manifest |
| [docs/report.md](docs/report.md) | What a run leaves behind: `summary.json`, contact sheets, crash reports |
| [docs/mutation-testing.md](docs/mutation-testing.md) | `mutate.py` operators, config, reports, how to read a score |
| [docs/prose-qa.md](docs/prose-qa.md) | The prose gates and the claims ledger |
| [docs/case-studies.md](docs/case-studies.md) | Does the mutation layer find real gaps? Measured on humanize, tenacity and swift-argument-parser |
| [docs/pypi.md](docs/pypi.md) | The pip package: the five commands, where the skills land, the release procedure |
| [examples/README.md](examples/README.md) | The Notes example app: build it, walk it, see the two planted defects reported |
| [docs/agents.md](docs/agents.md) | Attaching the kit to Claude Code (plugin), Codex, Cursor, Gemini CLI and any MCP client |
| [mcp/README.md](mcp/README.md) | The MCP server: nine tools, one stdio process, no dependencies |
| [skills/README.md](skills/README.md) | Installing the skills by hand |
| [ROADMAP.md](ROADMAP.md) | What comes next and the loop that keeps the kit honest |
| [RELEASING.md](RELEASING.md) | How a change becomes a tagged release |

## The Swift package

```swift
import XCTest
import QAKit

QA.snap("settings", in: self)                       // screenshot into the result bundle, named by variant
QA.audit(app, "settings", in: self)                 // Apple's accessibility audit, every issue in one report
QA.walk(app, "settings", in: self, deny: ["Delete"]) { QA.dismissTopmost(app) }   // snap, audit, sweep, scroll
QA.answerSystemAlert()                              // permission prompts, in the phone's language
QA.settle(app, on: screen, in: self) { relaunch() } // get back to a known screen, relaunch as the last resort
```

Requirements: iOS 17 or later for the accessibility audit, Xcode 15 or later to build, Xcode 27 or later for the
device variants in `device_qa.py`.

## Who runs it

- The author's app monorepo pins the kit as a submodule and runs every layer on a USB-attached test iPhone.
- [forge](https://github.com/younatics/forge), the multi-agent coding loop, runs the mutation and prose gates as merge
  authority on every task and gives its reviewer the qa-rigor principles.
- Gaps either of them hits come back here as a fix with a test and a tag (`RELEASING.md`).

## Status

Private beta. The three layers are in daily use on the author's apps and documents; the API and the manifest keys may
still change between minor versions until 1.0. See [CHANGELOG.md](CHANGELOG.md).

## License

MIT. See [LICENSE](LICENSE).
