Appropriate use
Use after requirements are sufficiently clear and before manual or automated execution begins.
Scope and intent
A controlled workflow specification
This page defines the behaviour, controls, evidence, and outputs required from the skill. It does not claim that a language model can enforce security or correctness by wording alone.
Non-goals
Do not use to approve unclear requirements, invent expected behaviour, or claim coverage for combinations that were not selected.
- 8
- versioned workflow phases
- 3
- explicit approval gates
- 5
- defined result states
- 8
- versioned output artifacts
Material changes to the source, environment, identity, approved actions, or expected behaviour invalidate dependent approvals and require revalidation.
Workflow
The 8 phases
Each phase has explicit inputs and outputs. Approval gates are hard stops bound to exact versions and hashes.
Phase 1: Input and revision validation Gate 1 · approve planning inputs
Resolve approved requirements, ticket revision, designs, API contracts, implementation diff when available, target environments, and supported platforms.
source versions · implementation status · environment · supported matrix
Phase 2: Risk model construction
Score customer impact, change complexity, data sensitivity, permission exposure, integration risk, reversibility, observability, and historical defect concentration.
likelihood · impact · detectability · recoverability · confidence
Phase 3: Coverage dimension discovery
Identify actors, roles, tenants, states, platforms, browsers, devices, locales, network conditions, data shapes, feature flags, and upgrade paths.
coverage dimensions · equivalence groups · mandatory combinations
Phase 4: Scenario design
Create atomic scenarios with purpose, priority, requirement trace, preconditions, data, steps, expected result, assertion type, evidence, and cleanup.
positive · negative · boundary · state transition · recovery · regression
Phase 5: Matrix optimisation
Use risk-based selection, pairwise reasoning where appropriate, and explicit equivalence assumptions to reduce combinations without hiding exclusions.
selected combinations · excluded combinations · equivalence rationale
Phase 6: Entry, stop, and exit criteria
Define what must be true before testing, when execution should stop, and how scenario outcomes map to completion or release readiness.
entry criteria · stop conditions · exit criteria · defect thresholds
Phase 7: Peer review and approval Gate 2 · approve plan
Present coverage, risk, assumptions, exclusions, test-data actions, estimated effort bands, and open questions for review.
coverage gaps · reviewer comments · revised scope
Phase 8: Freeze and publish plan Gate 3 · publish plan
Create a versioned plan with stable scenario IDs. Material requirement, build, or scope changes create a new revision rather than silently changing approved tests.
plan hash · revision · owner · change log · downstream references
Approval contract
Every gate approves an exact decision object
An approval cannot be reused after its environment, source, action plan, or approved document changes.
| Gate | Decision | Required binding |
|---|---|---|
| 1 | Gate 1 · approve planning inputs | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 2 | Gate 2 · approve plan | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 3 | Gate 3 · publish plan | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
Result model
Defined states prevent vague conclusions
Attempt outcomes remain separate from these workflow results. Errors, cancellations, and invalid context are never silently converted into a pass.
APPROVED
The plan is accepted for execution against the recorded scope and revision.
APPROVED WITH GAPS
Known, owned exclusions remain and are visible in the exit criteria.
REVISION REQUIRED
Coverage, traceability, data, evidence, or expected results need correction.
BLOCKED
Planning cannot complete because required requirements, environment, or ownership decisions are missing.
SUPERSEDED
A newer approved revision replaces this plan.
Evidence contract
Evidence must directly support the claim
Evidence has provenance, capture context, privacy classification, retention, and integrity metadata. A convenient artifact is not automatically sufficient proof.
| Claim | Primary evidence | Required validation |
|---|---|---|
| Requirement coverage | Traceability map | Every required behaviour links to at least one scenario or a justified exclusion. |
| Risk priority | Risk score and rationale | Priority is based on impact and likelihood, not test-author preference. |
| Platform selection | Coverage matrix | Included and excluded combinations are explicit. |
| Expected result | Approved requirement reference | No expected outcome is invented from implementation alone. |
| Evidence design | Assertion-to-evidence mapping | Evidence can directly support the intended conclusion. |
| Data plan | Creation and cleanup record | Ownership, side effects, and cleanup are defined before execution. |
Trust and safety
Controls that must be enforced outside the prompt
These controls belong in the runtime, connector permissions, sandbox, renderer, storage layer, and review process.
Runtime control
Stable scenario IDs
Scenario identity survives wording edits and is never reused for a different test.
Runtime control
Traceability
Required scenarios map to approved requirements, risks, or historical defects.
Runtime control
No fake completeness
Coverage percentage excludes out-of-scope dimensions rather than implying they were tested.
Runtime control
Change control
Material source changes invalidate or revise the plan.
Runtime control
Safe data planning
Destructive or external side effects require later execution approval.
Runtime control
Review separation
The plan author cannot silently self-approve when policy requires independent review.
Decision policy
Ordered outcomes, not averaged pass counts
The first applicable high-severity outcome takes precedence. A manual override is recorded separately and never rewrites the calculated result.
BLOCKED
Required planning inputs or decisions are missing.
REVISION REQUIRED
The plan is incomplete, unsafe, untraceable, or inconsistent.
APPROVED WITH GAPS
Execution may proceed with documented, accepted exclusions.
APPROVED
The plan is traceable, risk-based, executable, and appropriately scoped.
Known limitations
Uncertainty and coverage limits remain visible
The workflow documents what it cannot prove and what further evidence would change confidence.
- Limit 1. A test plan predicts useful coverage; it does not prove the product works.
- Limit 2. Pairwise reduction is unsuitable when particular combinations carry unique regulatory, security, or platform risk.
- Limit 3. Effort is expressed as bands unless reliable execution history exists.
- Limit 4. Unknown implementation details may require a later impact-analysis revision.
- Limit 5. Automation suitability and execution priority are related but separate decisions.
- Limit 6. Emergency work may use a reduced plan, but the omitted dimensions and accepted risk must be explicit.
Outputs
Versioned records for review and continuation
Human summaries are generated from validated structured records. They are not maintained as separate, drifting sources of truth.
plan-context.jsonVersioned output produced by the workflow.
risk-register.yamlVersioned output produced by the workflow.
coverage-matrix.csvVersioned output produced by the workflow.
test-plan.mdVersioned output produced by the workflow.
scenario-catalog.yamlVersioned output produced by the workflow.
data-plan.mdVersioned output produced by the workflow.
review-log.jsonlVersioned output produced by the workflow.
test-plan.htmlVersioned output produced by the workflow.
artifacts/qa/test-plan/<run-id>/ ├── plan-context.json ├── risk-register.yaml ├── coverage-matrix.csv ├── test-plan.md ├── scenario-catalog.yaml ├── data-plan.md ├── review-log.jsonl └── test-plan.html
Example
Example request and controlled output
Input
Create a test plan for QA-3302. Cover Admin and Member roles, Chrome and Samsung Tizen, failed API calls, upgrade behaviour, and regression around existing playlists.
Output excerpt
Plan status: REVISION REQUIRED Reason: The ticket does not define expected Member behaviour for editing an inherited playlist. Prepared: - 18 draft scenarios - 6 required P0/P1 scenarios - Chrome and Tizen coverage matrix - Network failure and upgrade scenarios Pending: Product decision for inherited playlist permissions.
Production readiness
Required before operational use
The written skill is only one layer. The repository, runtime, connectors, evidence storage, and publication path must implement these requirements.
- Required. Tool, filesystem, repository, command, secret, and network permissions are enforced outside the model.
- Required. Every source, environment, build, plan, and approval has a version or content hash.
- Required. Untrusted values are escaped, length-bounded, path-safe, and separated from executable instructions.
- Required. Attempt outcomes, scenario verdicts, workflow decisions, and publication status remain separate.
- Required. Raw evidence, publishable evidence, access control, retention, redaction, and deletion are defined.
- Required. Concurrent runs use unique IDs, locks, idempotency keys, and conflict-aware updates.
- Required. Material changes invalidate dependent approvals and outcomes.
- Required. The repository validator passes before the skill is considered production-ready.
In one line
/test-plan
Creates a reviewable test plan from approved requirements, implementation context, and known risk, with explicit scope, required scenarios, evidence, data, and exit criteria.