Appropriate use
Use during percentage rollouts, canary expansion, store deployments, firmware rollout, or controlled feature-flag release.
Scope and intent
A controlled workflow specification
This page defines the behaviour, controls, evidence, and outputs required from the skill. It does not claim that a language model can enforce security or correctness by wording alone.
Non-goals
Do not use without a baseline, stage identity, hold period, rollback owner, and metrics that can distinguish the new cohort from the control population.
- 8
- versioned workflow phases
- 3
- explicit approval gates
- 5
- defined result states
- 9
- versioned output artifacts
Material changes to the source, environment, identity, approved actions, or expected behaviour invalidate dependent approvals and require revalidation.
Workflow
The 8 phases
Each phase has explicit inputs and outputs. Approval gates are hard stops bound to exact versions and hashes.
Phase 1: Stage preflight Gate 1 · approve stage
Verify release candidate, rollout mechanism, current percentage, cohort definition, control group, stage start time, hold period, and rollback capability.
release ID · stage · cohort · control · start · rollback
Phase 2: Threshold and metric binding
Load approved success, warning, hold, and rollback thresholds for crashes, errors, playback, offline devices, latency, adoption, support cases, and business-critical signals.
metric definition · query · baseline window · threshold · owner
Phase 3: Data quality validation
Check telemetry freshness, sample size, version attribution, duplicate events, missing devices, clock skew, delayed ingestion, and dashboard changes.
freshness · completeness · attribution · confidence
Phase 4: Cohort and baseline comparison
Compare the rollout cohort against control, previous version, historical range, platform segments, geography, hardware, and network where material.
absolute change · relative change · confidence · segment outliers
Phase 5: Incident and qualitative review
Review support tickets, crash reports, customer messages, device offline patterns, logs, and known incidents that may not yet appear in aggregate metrics.
qualitative signals · correlated incidents · false-positive checks
Phase 6: Decision calculation
Apply ordered rules to data validity, rollback thresholds, warning thresholds, hold duration, unresolved incidents, and adoption progress.
invalid data · rollback · hold · investigate · continue
Phase 7: Stage review Gate 2 · approve stage action
Present metric evidence, anomalies, confidence, unresolved signals, and the calculated decision. Overrides remain separately recorded.
decision · evidence window · owner · next review time
Phase 8: Execute action and handoff Gate 3 · execute action
Record continue, hold, investigate, or rollback action idempotently, preserve the stage snapshot, and schedule the next required checkpoint.
action record · rollback confirmation · next stage · monitoring handoff
Approval contract
Every gate approves an exact decision object
An approval cannot be reused after its environment, source, action plan, or approved document changes.
| Gate | Decision | Required binding |
|---|---|---|
| 1 | Gate 1 · approve stage | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 2 | Gate 2 · approve stage action | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
| 3 | Gate 3 · execute action | Approval stores approver, role, timestamp, context hash, approved document hash, expiry, and invalidation state. |
Result model
Defined states prevent vague conclusions
Attempt outcomes remain separate from these workflow results. Errors, cancellations, and invalid context are never silently converted into a pass.
CONTINUE
Stage health meets approved criteria and the hold requirement is satisfied.
HOLD
No rollback threshold is breached, but time, adoption, sample, or warning signals are insufficient.
INVESTIGATE
Material anomalies require focused analysis before expansion.
ROLL BACK
A defined rollback threshold or critical qualitative signal is met.
INVALID DATA
Telemetry or cohort attribution is not trustworthy enough for a rollout decision.
Evidence contract
Evidence must directly support the claim
Evidence has provenance, capture context, privacy classification, retention, and integrity metadata. A convenient artifact is not automatically sufficient proof.
| Claim | Primary evidence | Required validation |
|---|---|---|
| Stage identity | Rollout platform snapshot | Percentage, cohort, release, and start time are immutable for the decision window. |
| Metric | Versioned query result | Definition, time window, baseline, sample, and freshness are recorded. |
| Device health | Online/offline and crash telemetry | New cohort is distinguishable from control. |
| Playback or core action | Success and failure rates | Event deduplication and delayed ingestion are considered. |
| Support signal | Ticket and incident references | Qualitative evidence is linked to version and cohort where possible. |
| Rollback | Action confirmation | Rollback completion and residual exposure are verified. |
Trust and safety
Controls that must be enforced outside the prompt
These controls belong in the runtime, connector permissions, sandbox, renderer, storage layer, and review process.
Runtime control
Predefined thresholds
Thresholds cannot be changed after seeing the data without a separately approved policy revision.
Runtime control
Cohort integrity
Devices switching cohorts or versions are accounted for.
Runtime control
Data freshness
Late telemetry cannot be mistaken for recovery.
Runtime control
Segment visibility
Aggregate health cannot hide a severe platform or hardware subgroup.
Runtime control
Override separation
A business decision to continue does not rewrite the calculated QA outcome.
Runtime control
Rollback verification
Issuing rollback is not completion; adoption and health must be confirmed afterward.
Decision policy
Ordered outcomes, not averaged pass counts
The first applicable high-severity outcome takes precedence. A manual override is recorded separately and never rewrites the calculated result.
INVALID DATA
Telemetry or cohort attribution cannot support a safe decision.
ROLL BACK
A critical or predefined rollback condition is met.
INVESTIGATE
Material anomalies require focused diagnosis.
HOLD
More time, adoption, or evidence is required.
CONTINUE
Health and hold criteria support expansion.
Known limitations
Uncertainty and coverage limits remain visible
The workflow documents what it cannot prove and what further evidence would change confidence.
- Limit 1. Small stages may lack statistical power for low-frequency failures.
- Limit 2. Support reports can arrive before telemetry and deserve separate weight.
- Limit 3. Telemetry instrumentation defects can mimic product improvement or degradation.
- Limit 4. A healthy early cohort may not represent later regions or hardware.
- Limit 5. Store and device update behaviour can delay adoption unpredictably.
- Limit 6. Rollback may not immediately remove already downloaded or cached versions.
Outputs
Versioned records for review and continuation
Human summaries are generated from validated structured records. They are not maintained as separate, drifting sources of truth.
stage-context.jsonVersioned output produced by the workflow.
threshold-policy.yamlVersioned output produced by the workflow.
metric-snapshot.jsonVersioned output produced by the workflow.
cohort-analysis.mdVersioned output produced by the workflow.
incident-review.mdVersioned output produced by the workflow.
stage-decision.jsonVersioned output produced by the workflow.
action-record.jsonVersioned output produced by the workflow.
rollback-verification.mdVersioned output produced by the workflow.
rollout-report.htmlVersioned output produced by the workflow.
artifacts/qa/staged-rollout-check/<run-id>/ ├── stage-context.json ├── threshold-policy.yaml ├── metric-snapshot.json ├── cohort-analysis.md ├── incident-review.md ├── stage-decision.json ├── action-record.json ├── rollback-verification.md └── rollout-report.html
Example
Example request and controlled output
Input
Check the PIXI OS 1.4.3 rollout at 20%. Compare offline rate, reboot loops, proxy-screen loops, support cases, and version adoption against 1.4.2.
Output excerpt
Rollout decision: INVESTIGATE Why: Overall health is within threshold, but devices offline longer than seven days show a threefold increase in proxy-screen loops. Action: Hold at 20%. Next: Segment by cached-content type and offline duration, reproduce with long-offline laboratory devices, and review again after six hours.
Production readiness
Required before operational use
The written skill is only one layer. The repository, runtime, connectors, evidence storage, and publication path must implement these requirements.
- Required. Tool, filesystem, repository, command, secret, and network permissions are enforced outside the model.
- Required. Every source, environment, build, plan, and approval has a version or content hash.
- Required. Untrusted values are escaped, length-bounded, path-safe, and separated from executable instructions.
- Required. Attempt outcomes, scenario verdicts, workflow decisions, and publication status remain separate.
- Required. Raw evidence, publishable evidence, access control, retention, redaction, and deletion are defined.
- Required. Concurrent runs use unique IDs, locks, idempotency keys, and conflict-aware updates.
- Required. Material changes invalidate dependent approvals and outcomes.
- Required. The repository validator passes before the skill is considered production-ready.
In one line
/staged-rollout-check
Evaluates each rollout stage using version adoption, product health, device state, support signals, cohort comparison, and predefined rollback thresholds.