---
type: Axis
title: recovery (failure-handling axis)
description: The recovery/failure-handling architecture axis — how the system responds to failure of an operation (fail-fast → retry → circuit-breaker → compensating-saga). A system-singleton axis whose commitment surface is durable runtime state at the upper rungs; worked example is the interceptor compensating chains (#294).
tags: [axis, architecture, recovery, failure, saga, interceptor, rfc-75, deep]
timestamp: 2026-06-18T00:00:00Z
---

# Recovery (failure-handling axis)

> **DEEP axis (5A.2).** Full §6 depth with a worked example from the skill's own
> history — the interceptor compensating chains (#294).

## Concept

Recovery is the **failure-handling** axis (RFC #75 §3): how the system responds
when an operation fails partway. The ladder runs fail-fast (surface the error,
do nothing else) → retry (re-attempt transient failures with backoff) →
circuit-breaker (stop hammering a failing dependency; degrade) →
compensating-saga (undo the already-committed steps of a partial multi-step
operation). Its **commitment surface is durable runtime state** at the upper
rungs (§4): a saga must *remember* what it did to undo it; a breaker holds state
about a dependency's health. Its **scope is system-singleton** (§5): the failure
posture is chosen for the system.

## Why it is an axis (discriminator test — §2)

1. **Ordered tiers** — each rung handles a harder failure shape.
2. **Cross-cutting bundle** — a rung dictates the error-propagation pattern, the
   retry/backoff policy, the breaker state store, the compensation registry, and
   the failure-injection tests.
3. **Asymmetric migration cost** — retrofitting compensation onto a system that
   commits partial work without recording it is expensive: you must add the
   durable record of what-to-undo after the fact.
4. **Requirement-keyed selection** — falsifiable probes pick the rung.

## Schema view (the §6 entry — validated by `axis_catalog`)

```yaml
id: recovery
kind: architecture-axis
scope: system-singleton
selection:
  probes:
    - "Is surfacing the error and stopping acceptable? → fail-fast."
    - "Are failures often transient (network blip, lock contention)? → retry with backoff."
    - "Can a failing dependency be hammered into a worse state? → circuit-breaker + degrade."
    - "Does a multi-step operation commit work that must be UNDONE if a later step fails? → compensating-saga."
    - "Is there partial committed work with no recorded way to undo it? → under-modeled; you need the saga's compensation record."
composes_with:
  - "conflict resolution = consistency × collaboration (recovery interacts with both)"
applies_when: []
tiers:
  - tier: fail-fast
    commitment_delta:
      surface: durable-runtime-state
      delta: "an operation that fails surfaces the error and stops; no automatic remediation. State is left wherever the failure found it."
    admin_tool: "error log / alert surface."
    enforcement:
      - "errors propagate to the caller; no silent swallow."
    tests:
      - "a failing step raises; the caller sees the error."
    forecloses: "automatic recovery; the operator remediates."
    migration_cost_to_next: "low — wrap transient-prone calls in bounded retry."
  - tier: retry
    commitment_delta:
      surface: durable-runtime-state
      delta: "transient failures are re-attempted with bounded backoff; the operation is made idempotent so a retry is safe."
    admin_tool: "retry-attempt metrics; dead-letter for exhausted retries."
    enforcement:
      - "retried operations are idempotent; retries are bounded (count + backoff)."
    tests:
      - "a transient failure succeeds on retry; a permanent failure exhausts and surfaces."
    forecloses: "treating every failure as fatal; but also masks a real outage if unbounded."
    migration_cost_to_next: "medium — track dependency health to stop retrying a down dependency."
  - tier: circuit-breaker
    commitment_delta:
      surface: durable-runtime-state
      delta: "a breaker tracks a dependency's recent health; on sustained failure it opens (fail fast + degrade) and probes for recovery before closing."
    admin_tool: "breaker-state dashboard; manual trip/reset."
    enforcement:
      - "calls through the breaker respect its open/half-open/closed state; a degraded path exists."
    tests:
      - "sustained failure opens the breaker; recovery half-opens then closes; the degraded path serves while open."
    forecloses: "naive infinite retry against a down dependency."
    migration_cost_to_next: "high — record committed steps so a partial multi-step op can be undone."
  - tier: compensating-saga
    commitment_delta:
      surface: durable-runtime-state
      delta: "a multi-step operation records each committed step + its compensating (undo) action; on a late failure it runs the compensations in reverse. The what-to-undo record is durable."
    admin_tool: "saga-state inspector; manual compensation replay."
    enforcement:
      - "every forward step registers its compensation before the next step; a failure triggers reverse compensation."
    tests:
      - "a failure at step N runs compensations N-1..1 and leaves the system consistent; compensation is idempotent."
    forecloses: "fire-and-forget multi-step writes; every step is paired with its undo."
    migration_cost_to_next: "n/a — top of the ladder."
examples:
  - "The skill's own climb: the recording pipeline's interceptor chains gained compensating behavior (#294) — when a multi-backend write commits to one forge but fails on the next, the chain runs compensating actions (replay_pending / pending-op rollback) rather than leaving a half-recorded turn. The pending-op queue is the durable what-to-undo / what-to-finish record; replay_pending is the saga's resume. Most of the skill sits at retry (multi-backend writes are idempotent + retried); the interceptor compensation is the saga rung where partial commits must be reconciled."
```

## Selection procedure (§8)

Bidirectional check; chosen once for the system's failure posture. Recorded in
the system architecture record; a multi-step write that commits partial work
without a compensation record is a systems-analysis D-finding.

## Composition (§5)

Recovery interacts with **consistency × collaboration** (conflict resolution):
a saga's compensation must respect the consistency model when undoing across
writers. Rule authored in 5A.3.
