/feature-qa

Scope and intent

A controlled workflow specification

This page defines the behaviour, controls, evidence, and outputs required from the skill. It does not claim that a language model can enforce security or correctness by wording alone.

Use when

Appropriate use

Use for meaningful product changes that require code-aware impact analysis, planned execution, traceable evidence, defect triage, and a release recommendation.

Do not use when

Non-goals

Do not use for one-line copy edits, emergency production operations, or any environment where deployment identity and test-data ownership cannot be verified.

8: versioned workflow phases
6: explicit approval gates
6: defined result states
13: versioned output artifacts

Material changes to the source, environment, identity, approved actions, or expected behaviour invalidate dependent approvals and require revalidation.

Workflow

The 8 phases

Each phase has explicit inputs and outputs. Approval gates are hard stops bound to exact versions and hashes.

Phase 1: Secure preflight and deployment attestation Gate 1 · approve environment

Resolve the ticket, PR, repositories, environment, authenticated role, feature flags, browser context, and deployment manifest. Validate that the deployed artifact contains the approved source commits.

ticket version · PR head/base · source commit map · artifact digest · environment identity · role · browser · viewport

Phase 2: Requirements and change-impact analysis

Read acceptance criteria, diff, related code, routes, types, permissions, existing tests, dependencies, migrations, and observability. Record contradictions as requirement gaps instead of inventing behaviour.

direct impact · indirect impact · security · accessibility · compatibility · migration · observability

Phase 3: Risk-based test plan Gate 2 · approve scope

Create numbered scenarios with priority, preconditions, steps, expected results, assertion type, evidence type, test matrix, stop conditions, required status, and explicit exclusions.

P0–P3 · positive · negative · edge · regression · roles · browsers · devices · account states

Phase 4: Data, side-effect, and execution authorization Gate 3 · approve actions

List every create, update, delete, feature-flag change, external notification, stub, mock, webhook, download, and cleanup action. Bind approval to the hashed data plan and run context.

idempotency keys · ownership IDs · test sinks · no hidden writes · reversible changes

Phase 5: Execution and assertion-specific evidence

Run required scenarios in priority order. Record every attempt separately and capture evidence appropriate to the assertion rather than treating screenshots as universal proof.

screenshots · traces · video · network · logs · timing · persistence checks · first failure retained

Phase 6: Finding classification and defect confirmation Gate 4 · confirm finding

Classify non-pass results as product defect, environment failure, automation failure, data failure, requirement gap, third-party failure, security concern, or unknown. Search for candidate duplicates without suppressing legitimate findings.

reproduction attempted · confidence recorded · candidate duplicates reviewed

Phase 7: Regression protection Gate 5 · approve test change

Recommend the lowest reliable test layer and validate generated tests through red-to-green proof or an approved alternative when the broken state cannot be safely recreated.

unit · component · contract · API · integration · Playwright · stable selectors · isolated data

Phase 8: Decision, publication, and cleanup Gate 6 · review and publish

Revalidate deployment and approvals, calculate the ordered release outcome, review evidence, publish one idempotent ticket update, render the report bundle, and clean only verified owned resources.

context revalidation · override recorded separately · cleanup residuals listed · resumable checkpoints

Approval contract

Every gate approves an exact decision object

An approval cannot be reused after its environment, source, action plan, or approved document changes.

Approval gates and binding requirements
Gate	Decision	Required binding
1	Gate 1 · approve environment	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
2	Gate 2 · approve scope	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
3	Gate 3 · approve actions	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
4	Gate 4 · confirm finding	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
5	Gate 5 · approve test change	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
6	Gate 6 · review and publish	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.

Result model

Defined states prevent vague conclusions

Attempt outcomes remain separate from these workflow results. Errors, cancellations, and invalid context are never silently converted into a pass.

PASS

All required steps ran and the approved expected result was directly observed with valid evidence.

FAIL

The controlled scenario ran and the actual result contradicted the approved expectation.

INCONCLUSIVE

Execution completed, but expectation or evidence remains genuinely ambiguous.

BLOCKED

A verified prerequisite outside the scenario prevented completion.

NOT TESTED

The scenario was not attempted and the reason is recorded.

FLAKY

Controlled attempts produced inconsistent product outcomes. It is never converted to PASS.

Evidence contract

Evidence must directly support the claim

Evidence has provenance, capture context, privacy classification, retention, and integrity metadata. A convenient artifact is not automatically sufficient proof.

Required evidence by claim type
Claim	Primary evidence	Required validation
Visual state	Screenshot	Asserted element and relevant context are visible, readable, and timestamped.
Navigation or transition	Trace or video plus URL history	Origin, destination, intermediate state, and timing are preserved.
API behaviour	Request and response capture	Method, endpoint, status, schema, and approved relevant headers match.
Permissions	Role attestation plus UI and API evidence	Tenant, ownership, operation, and role are independently verified.
Persistence	Approved API or read-only data verification	State survives reload or a new session without hidden caching.
Performance	Measured timing samples	Threshold, sample count, variance, browser, build, and environment are recorded.
Long-running behaviour	Video, runtime logs, health samples	Observation duration and interruptions are explicit.
Runtime stability	Console, crash, and application logs	New errors are separated from a versioned, expiring baseline.

Trust and safety

Controls that must be enforced outside the prompt

These controls belong in the runtime, connector permissions, sandbox, renderer, storage layer, and review process.

Runtime control

Tool-enforced least privilege

The runtime, not the prompt, enforces filesystem, command, repository, secret, and network permissions.

Runtime control

Version-bound approvals

Every approval stores the approver, timestamp, role, document hash, context hash, expiry, and invalidation reason.

Runtime control

Continuous context validation

Deployment, role, tenant, feature flags, and relevant configuration are rechecked before execution, resume, decision, and publication.

Runtime control

Output sanitisation

Untrusted values are HTML-escaped, Markdown-safe, length-bounded, link-validated, and never used directly as paths.

Runtime control

Tamper-evident ledger

Events are sequenced and hash-chained; evidence manifests include capture context and are signed when stronger assurance is required.

Runtime control

Evidence privacy

Sensitive headers are filtered before storage; raw and publishable evidence have separate access, retention, and redaction policies.

Runtime control

Concurrency safety

Run locks, idempotency keys, unique IDs, and compare-and-update operations prevent silent overwrites.

Runtime control

Data ownership

Real returned resource IDs are recorded immediately; cascade impact and cleanup uncertainty require explicit review.

Decision policy

Ordered outcomes, not averaged pass counts

The first applicable high-severity outcome takes precedence. A manual override is recorded separately and never rewrites the calculated result.

INVALID RUN

Deployment identity, approval binding, environment integrity, or evidence integrity cannot be established.

REJECT

An unresolved release-blocking product, security, data-integrity, or compliance defect exists.

BLOCKED

Any required scenario is blocked, inconclusive, not tested, flaky, or ended in an execution error.

CONDITIONAL PASS

All required scenarios passed, but explicitly accepted optional gaps or residual risks remain.

PASS

All required scenarios passed, no unaccepted release-blocking risk remains, and cleanup state is acceptable.

Known limitations

Uncertainty and coverage limits remain visible

The workflow documents what it cannot prove and what further evidence would change confidence.

Limit 1. A preview may use a synthetic merge commit or several repositories; validate a deployment manifest rather than comparing only one SHA.
Limit 2. Some defects cannot be reproduced safely. Reproduction is attempted, but strong original evidence may still justify a finding.
Limit 3. A short observation cannot establish long-term stability; duration and confidence remain visible.
Limit 4. Database, infrastructure, and security checks require explicitly approved read-only mechanisms or isolated environments.
Limit 5. Third-party availability and rate limits are classified separately from product behaviour.
Limit 6. Generated regression tests remain proposed changes until reviewed and validated in the target repository.

Outputs

Versioned records for review and continuation

Human summaries are generated from validated structured records. They are not maintained as separate, drifting sources of truth.

run-context.json