Appropriate use
Use after releases, incidents, major defects, automation migrations, or recurring QA pain.
Scope and intent
A controlled workflow specification
This page defines the behaviour, controls, evidence, and outputs required from the skill. It does not claim that a language model can enforce security or correctness by wording alone.
Non-goals
Do not use for blame, performance ranking, unsupported storytelling, or creating actions without owners and validation measures.
- 8
- versioned workflow phases
- 3
- explicit approval gates
- 6
- defined result states
- 9
- versioned output artifacts
Material changes to the source, environment, identity, approved actions, or expected behaviour invalidate dependent approvals and require revalidation.
Workflow
The 8 phases
Each phase has explicit inputs and outputs. Approval gates are hard stops bound to exact versions and hashes.
Phase 1: Scope and psychological-safety preflight Gate 1 · approve retrospective scope
Confirm event, time window, participants, facilitator, data sources, privacy, no-blame principle, and what decisions the retrospective may make.
event ID · window · participants · data boundary
Phase 2: Evidence timeline
Build a factual sequence of requirements, code changes, test planning, execution, defects, decisions, rollout, incidents, recovery, and customer signals.
event · timestamp · source · fact · uncertainty
Phase 3: Outcome and escape analysis
Identify what succeeded, what failed, escaped defects, false alarms, delayed detection, repeated manual work, missing coverage, and operational cost.
impact · detection point · escape path · recovery
Phase 4: Contributing-system analysis
Examine requirements, architecture, ownership, environment, data, automation, observability, release controls, communication, and incentives without reducing the event to one person.
contributing condition · evidence · interaction
Phase 5: Control effectiveness review
Assess which prevention, detection, containment, and recovery controls worked, failed, were missing, or produced false confidence.
control · expected · actual · gap · confidence
Phase 6: Action design
Create specific preventive, detective, recovery, or learning actions with owner, due condition, priority, success measure, cost, and dependency.
action · owner · measure · due · dependency
Phase 7: Review and commitment Gate 2 · approve actions
Present the narrative, uncertainties, proposed actions, rejected actions, and ownership. Separate consensus from unresolved disagreement.
accepted · rejected · unresolved · owner
Phase 8: Publish and follow-up Gate 3 · publish
Publish the retrospective, create linked work items, schedule outcome reviews, and close actions only after the defined success measure is observed.
retro ID · action tickets · review date · evidence
Approval contract
Every gate approves an exact decision object
An approval cannot be reused after its environment, source, action plan, or approved document changes.
| Gate | Decision | Required binding |
|---|---|---|
| 1 | Gate 1 · approve retrospective scope | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 2 | Gate 2 · approve actions | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 3 | Gate 3 · publish | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
Result model
Defined states prevent vague conclusions
Attempt outcomes remain separate from these workflow results. Errors, cancellations, and invalid context are never silently converted into a pass.
EFFECTIVE
The control or practice reduced risk as intended.
PARTIALLY EFFECTIVE
The control helped but had material limits or gaps.
INEFFECTIVE
The control did not prevent, detect, contain, or recover as expected.
MISSING
No relevant control existed for the observed risk.
FALSE CONFIDENCE
The control appeared healthy while failing to measure the real risk.
NOT ENOUGH EVIDENCE
The retrospective cannot support a confident conclusion.
Evidence contract
Evidence must directly support the claim
Evidence has provenance, capture context, privacy classification, retention, and integrity metadata. A convenient artifact is not automatically sufficient proof.
| Claim | Primary evidence | Required validation |
|---|---|---|
| Timeline | Timestamped authoritative sources | Facts and recollections are distinguished. |
| Escape | Requirement-to-release trace | The point where the risk could have been prevented or detected is evidenced. |
| Impact | Customer and operational data | Scope and recovery cost are not exaggerated. |
| Control | Expected and actual behaviour | The control's purpose and limitation are explicit. |
| Action | Owned measurable work item | Completion criteria measure outcome, not document creation alone. |
| Follow-up | Post-action evidence | An action closes only after effectiveness is checked. |
Trust and safety
Controls that must be enforced outside the prompt
These controls belong in the runtime, connector permissions, sandbox, renderer, storage layer, and review process.
Runtime control
No blame
Analysis focuses on system conditions, decisions, and controls while preserving accountability for actions.
Runtime control
Evidence over memory
Recollections are useful but labelled and checked against records.
Runtime control
Action limit
Prefer a small number of high-value owned actions over a long unprioritised list.
Runtime control
No cosmetic closure
Training, documentation, or automation is not assumed effective without a measure.
Runtime control
Sensitive handling
Customer, security, incident, and personnel data follow the correct audience policy.
Runtime control
Follow-through
Actions have review dates and cannot disappear after publication.
Decision policy
Ordered outcomes, not averaged pass counts
The first applicable high-severity outcome takes precedence. A manual override is recorded separately and never rewrites the calculated result.
INSUFFICIENT EVIDENCE
Collect missing records before drawing material conclusions.
SYSTEMIC RISK
High-impact missing or false-confidence controls require priority action.
IMPROVEMENT PLAN
Owned actions address material but manageable weaknesses.
HEALTHY WITH LEARNING
Controls were broadly effective and refinements are recorded.
Known limitations
Uncertainty and coverage limits remain visible
The workflow documents what it cannot prove and what further evidence would change confidence.
- Limit 1. Retrospectives are vulnerable to hindsight bias.
- Limit 2. Not every incident justifies a new automated test or process gate.
- Limit 3. Actions can create cost or delay elsewhere and need trade-off review.
- Limit 4. Some outcomes depend on external vendors or rare conditions.
- Limit 5. A single event may not prove a general pattern.
- Limit 6. The retrospective needs later evidence to show that actions worked.
Outputs
Versioned records for review and continuation
Human summaries are generated from validated structured records. They are not maintained as separate, drifting sources of truth.
retro-context.jsonVersioned output produced by the workflow.
timeline.jsonlVersioned output produced by the workflow.
outcome-analysis.mdVersioned output produced by the workflow.
contributing-systems.yamlVersioned output produced by the workflow.
control-review.csvVersioned output produced by the workflow.
actions.yamlVersioned output produced by the workflow.
decision-log.jsonlVersioned output produced by the workflow.
follow-up-plan.mdVersioned output produced by the workflow.
retrospective.htmlVersioned output produced by the workflow.
artifacts/qa/qa-retrospective/<run-id>/ ├── retro-context.json ├── timeline.jsonl ├── outcome-analysis.md ├── contributing-systems.yaml ├── control-review.csv ├── actions.yaml ├── decision-log.jsonl ├── follow-up-plan.md └── retrospective.html
Example
Example request and controlled output
Input
Run a QA retrospective for the release where a webOS black-screen issue escaped to customers. Focus on requirements, platform coverage, automation, rollout monitoring, and support detection.
Output excerpt
Retrospective outcome: SYSTEMIC RISK Key finding: The release smoke suite treated webOS 6 as one equivalence group, although laboratory evidence already showed model-specific browser differences. False-confidence control: A single representative device passed and was reported as webOS 6 coverage. Actions: 1. Split the platform matrix by browser engine and model family. 2. Add repeat-launch playback coverage. 3. Add model-segmented rollout telemetry. 4. Review effectiveness after the next two webOS releases.
Production readiness
Required before operational use
The written skill is only one layer. The repository, runtime, connectors, evidence storage, and publication path must implement these requirements.
- Required. Tool, filesystem, repository, command, secret, and network permissions are enforced outside the model.
- Required. Every source, environment, build, plan, and approval has a version or content hash.
- Required. Untrusted values are escaped, length-bounded, path-safe, and separated from executable instructions.
- Required. Attempt outcomes, scenario verdicts, workflow decisions, and publication status remain separate.
- Required. Raw evidence, publishable evidence, access control, retention, redaction, and deletion are defined.
- Required. Concurrent runs use unique IDs, locks, idempotency keys, and conflict-aware updates.
- Required. Material changes invalidate dependent approvals and outcomes.
- Required. The repository validator passes before the skill is considered production-ready.
In one line
/qa-retrospective
Reviews a completed release, incident, project, or testing cycle to identify escaped-risk patterns, process failures, effective controls, and owned preventive actions.