/incident-qa

Scope and intent

A controlled workflow specification

This page defines the behaviour, controls, evidence, and outputs required from the skill. It does not claim that a language model can enforce security or correctness by wording alone.

Use when

Appropriate use

Use during production incidents, severe customer-impacting failures, widespread platform regressions, or urgent mitigation verification.

Do not use when

Non-goals

Do not use to replace the incident commander, perform unapproved production changes, expose restricted customer data, or speculate publicly about cause.

8: versioned workflow phases
4: explicit approval gates
5: defined result states
10: versioned output artifacts

Material changes to the source, environment, identity, approved actions, or expected behaviour invalidate dependent approvals and require revalidation.

Workflow

The 8 phases

Each phase has explicit inputs and outputs. Approval gates are hard stops bound to exact versions and hashes.

Phase 1: Incident role and boundary preflight Gate 1 · approve investigation boundary

Confirm incident ID, commander, QA role, communication channel, affected services, production access policy, data restrictions, and approved test environments.

incident owner · QA owner · severity · access · communication

Phase 2: Timeline and symptom normalisation

Build a timestamped view of first detection, deployments, config changes, alerts, customer reports, mitigations, and current state. Separate facts from hypotheses.

fact · source · timestamp · confidence · hypothesis

Phase 3: Impact and cohort mapping

Identify affected versions, platforms, tenants, regions, roles, content types, networks, and time windows, plus known healthy comparisons.

affected cohort · unaffected cohort · frequency · recovery

Phase 4: Reproduction and comparison plan Gate 2 · approve actions

Design safe tests in staging, replicas, isolated production test accounts, or customer-provided evidence. Avoid adding load or changing customer state without approval.

safe environment · comparison · expected signature · stop condition

Phase 5: Investigation execution

Run controlled comparisons, collect logs and traces, validate failure signatures, and update the timeline. Preserve negative evidence and ruled-out hypotheses.

attempt ledger · evidence · hypothesis status · environment health

Phase 6: Mitigation verification

Verify whether rollback, config change, cache clear, restart, feature flag, or traffic shift removes the impact without introducing new failures.

before/after · residual impact · propagation · rollback readiness

Phase 7: Incident QA assessment Gate 3 · confirm assessment

State current reproducibility, affected scope, mitigation confidence, remaining risk, monitoring signals, and whether the incident can move toward recovery.

confirmed · probable · unresolved · residual risk

Phase 8: Recovery validation and handoff Gate 4 · publish handoff

Validate recovery criteria, provide follow-up regression and defect work, preserve the incident evidence bundle, and hand unresolved items to normal workflows.

recovery evidence · follow-up tickets · retrospective inputs

Approval contract

Every gate approves an exact decision object

An approval cannot be reused after its environment, source, action plan, or approved document changes.

Approval gates and binding requirements
Gate	Decision	Required binding
1	Gate 1 · approve investigation boundary	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
2	Gate 2 · approve actions	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
3	Gate 3 · confirm assessment	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.
4	Gate 4 · publish handoff	Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state.

Result model

Defined states prevent vague conclusions

Attempt outcomes remain separate from these workflow results. Errors, cancellations, and invalid context are never silently converted into a pass.

ACTIVE IMPACT

Controlled evidence or current telemetry confirms ongoing customer impact.

MITIGATED

The immediate impact is reduced, but residual risk or incomplete propagation remains.

RECOVERED

Approved recovery criteria are satisfied across the required observation window.

UNCONFIRMED

Reports exist but current evidence cannot confirm the failure signature.

INVALID EVIDENCE

Data provenance, time, environment, or attribution is unreliable.

Evidence contract

Evidence must directly support the claim

Evidence has provenance, capture context, privacy classification, retention, and integrity metadata. A convenient artifact is not automatically sufficient proof.

Required evidence by claim type
Claim	Primary evidence	Required validation
Timeline event	Authoritative timestamped source	Deployment, alert, customer, and mitigation times are distinguished.
Customer impact	Telemetry or approved customer evidence	Affected cohort and privacy handling are recorded.
Reproduction	Attempt ledger and trace	Laboratory and production observations are clearly separated.
Mitigation	Before-and-after metrics and functional checks	Propagation delay and residual exposure are included.
Recovery	Sustained health window	Required signals remain healthy for the agreed duration.
Hypothesis	Evidence-for and evidence-against table	Unproven cause remains labelled as hypothesis.

Trust and safety

Controls that must be enforced outside the prompt

These controls belong in the runtime, connector permissions, sandbox, renderer, storage layer, and review process.

Runtime control

Command hierarchy

The skill advises the incident commander and never executes production action without approval.

Runtime control

Load safety

Investigation cannot worsen the incident through unbounded retries or traffic.

Runtime control

Restricted evidence

Customer and production data follow incident-specific access and retention rules.

Runtime control

Fact-hypothesis separation

Every update distinguishes observed fact, inference, and proposed experiment.

Runtime control

Mitigation is not root cause

A successful rollback or restart does not prove why the incident occurred.

Runtime control

Recovery window

A temporary improvement is not RECOVERED until defined criteria and duration pass.

Decision policy

Ordered outcomes, not averaged pass counts

The first applicable high-severity outcome takes precedence. A manual override is recorded separately and never rewrites the calculated result.

INVALID EVIDENCE

Current data cannot support an incident QA conclusion.

ACTIVE IMPACT

Customer impact remains or mitigation failed.

UNCONFIRMED

Impact reports require more evidence or controlled comparison.

MITIGATED

Immediate impact is reduced with residual monitoring or action required.

RECOVERED

Recovery criteria pass for the required window.

Known limitations

Uncertainty and coverage limits remain visible

The workflow documents what it cannot prove and what further evidence would change confidence.

Limit 1. Incident conditions change faster than normal QA evidence.
Limit 2. Production access may be intentionally limited.
Limit 3. Customer environments can differ from laboratory reproduction.
Limit 4. Telemetry delays can obscure the timing of recovery.
Limit 5. Mitigations may hide the failure without resolving the cause.
Limit 6. Root-cause analysis and long-term prevention continue after recovery.

Outputs

Versioned records for review and continuation

Human summaries are generated from validated structured records. They are not maintained as separate, drifting sources of truth.

incident-context.json