QAKit

The QA agent kit. Procedures an AI coding agent follows to test like a QA engineer, the tools that run the checks, and the evidence format a release verdict cites. Three layers, one repo:

QAKit grew out of shipping several App Store apps as a solo developer whose QA team is an agent. Every check in it was added after a defect that green tests let through: a >= that became > at every grading boundary with 13 mutants surviving, a review note over the 4,000-character limit, a clipped caption nobody looked at, a What's New that still announced the first release in version 1.1.

See it

QA.walk on a real iPhone: the onboarding pages, the home screen at every scroll position, every visible button tapped once and screenshotted, the accessibility audit per screen. These 29 frames are one walkthrough test's attachments, assembled by tools/shots_gif.py from the result bundle (the app is the author's, live on the App Store).

The same walk in three variants: device_qa.py test --variant default --variant dark --variant large-text switches the phone's appearance and text size between runs. These nine frames are the examples/Notes app's list, settings and search screens from one such run.

What makes it different

• Systematic, not random. QA.sweepButtons taps every visible button on every screen at every scroll position, screenshots what opened, checks the app is still in front and goes back. Monkey testers tap at random; agents reviewing a contact sheet need every control seen once.
• The whole phone changes, not the app. device_qa.py test --variant dark switches the device to dark mode, large accessibility text, a simulated location or VoiceOver through Xcode 27's devicectl, runs the suite, and restores iOS defaults. System alerts and other apps render the same way the user will see them.
• Apple's accessibility audit on every screen, collected into one report per screen instead of stopping at the first issue, with an allow list for reviewed exceptions.
• Test strength is measured. Coverage says a line ran. Mutation testing says whether a test would notice if the line were wrong. The gate is "every mutant on a changed line is dead or explained".
• Prose gets the same treatment. Deterministic gates, then a claims ledger where every number is traced to a quote, then a cold read by a reader who did not write it.
• Written for agents. The skills are SKILL.md files a coding agent loads. The protocol has 180 checks with IDs, pass criteria, methods and sources, so an agent can cite what it did and what it skipped.

Quick start

pip install qa-agent-kit                           # qakit-mutate, qakit-prose, qakit-device, qakit-release-checks, qakit-install-skills
# or from source
git clone https://github.com/younatics/QAKit.git
cd QAKit && python3 -m pytest -q tests          # 150 tests, no dependency beyond pytest

# mutation testing on any project
python3 tools/mutate.py --root ../myproject --source 'src/**/*.py' --test '{orig}/.venv/bin/python -m pytest -x -q'

# prose
python3 tools/prose_check.py ../memo.md --claims claims.csv

# an Apple app on a USB-attached test iPhone (macOS, Xcode 27)
python3 tools/device_qa.py doctor --manifest ../myapp/release/testing/device_qa.json
python3 tools/device_qa.py test --manifest ../myapp/release/testing/device_qa.json --suite qa --variant dark

Give the procedures to your agent, whichever one you use:

claude plugin marketplace add younatics/QAKit && claude plugin install qakit@younatics   # Claude Code: skills + MCP tools
python3 tools/install_skills.py --target claude --scope project      # or codex, cursor, gemini, all; --scope user
claude mcp add qakit -- python3 /path/to/qakit/mcp/server.py        # any MCP client: doctor, test, shots, crashes, release checks, mutation, prose, verdict

Then read docs/getting-started.md. AGENTS.md tells any agent which skill to load for which job.

Documentation

The Swift package

import XCTest
import QAKit

QA.snap("settings", in: self)                       // screenshot into the result bundle, named by variant
QA.audit(app, "settings", in: self)                 // Apple's accessibility audit, every issue in one report
QA.walk(app, "settings", in: self, deny: ["Delete"]) { QA.dismissTopmost(app) }   // snap, audit, sweep, scroll
QA.answerSystemAlert()                              // permission prompts, in the phone's language
QA.settle(app, on: screen, in: self) { relaunch() } // get back to a known screen, relaunch as the last resort

Requirements: iOS 17 or later for the accessibility audit, Xcode 15 or later to build, Xcode 27 or later for the device variants in device_qa.py.

Who runs it

• The author's app monorepo pins the kit as a submodule and runs every layer on a USB-attached test iPhone.
• forge, the multi-agent coding loop, runs the mutation and prose gates as merge authority on every task and gives its reviewer the qa-rigor principles.
• Gaps either of them hits come back here as a fix with a test and a tag (RELEASING.md).

Status

Private beta. The three layers are in daily use on the author's apps and documents; the API and the manifest keys may still change between minor versions until 1.0. See CHANGELOG.md.

License

MIT. See LICENSE.