---
type: Axis
title: observability (introspection axis)
description: The observability architecture axis — how much of the running system's behavior is externally recorded (none → logging → metrics → tracing). A system-singleton axis whose commitment surface is recorded-history-in-an-external-store — the unrecorded past is unrecoverable; worked example is the daemon alert/SSE surface.
tags: [axis, architecture, observability, logging, metrics, tracing, rfc-75, deep]
timestamp: 2026-06-18T00:00:00Z
---

# Observability (introspection axis)

> **DEEP axis (5A.2).** Full §6 depth with a worked example from the skill's own
> history — the daemon alert / SSE / health surface. Observability is the axis
> that proves "schema" is not required for an axis (§4): it touches no
> persistent-data schema, yet its commitment is unrecoverable — the same
> foreclosure shape as temporality, on a different surface.

## Concept

Observability is the **introspection** axis (RFC #75 §3): how much of the running
system's behavior is recorded somewhere an operator can later read. The ladder
runs none (the system is a black box) → logging (event records, human-readable)
→ metrics (aggregated counters/gauges/histograms over time) → tracing (causally
linked spans across a request's path). Its **commitment surface is
recorded-history** in an external store (§4): **the unrecorded past is
unrecoverable** — you cannot reconstruct a metric for a window you never
recorded, exactly as you cannot backfill a `created_at` you never stamped. Its
**scope is system-singleton** (§5).

## Why it is an axis (discriminator test — §2)

1. **Ordered tiers** — each rung records strictly more about behavior.
2. **Cross-cutting bundle** — a rung dictates the emission points, the
   collection/store, the query surface, the retention policy, and the tests.
3. **Asymmetric migration cost** — recorded-history foreclosure: a window you
   didn't instrument is gone; adding tracing later cannot reconstruct past
   request causality.
4. **Requirement-keyed selection** — falsifiable probes pick the rung.

## Schema view (the §6 entry — validated by `axis_catalog`)

```yaml
id: observability
kind: architecture-axis
scope: system-singleton
selection:
  probes:
    - "Is post-hoc diagnosis never needed (a pure black box)? → none."
    - "Must an operator reconstruct WHAT happened from event records? → logging."
    - "Must trends/rates/aggregates over time be answerable (how often, how slow)? → metrics."
    - "Must a single request's causal path across components be reconstructable? → tracing."
    - "Was a window never instrumented? → that history is unrecoverable (the recorded-history foreclosure)."
composes_with:
  - "audit log = principal × temporality × authorization (observability is the operational sibling)"
applies_when: []
tiers:
  - tier: none
    commitment_delta:
      surface: recorded-history
      delta: "the system records nothing about its own behavior; it is a black box. Diagnosis is by reproduction only."
    admin_tool: "none."
    enforcement:
      - "no emission points; nothing to leak."
    tests:
      - "n/a (nothing recorded)."
    forecloses: "any post-hoc question about past behavior."
    migration_cost_to_next: "low — add structured event logging at key points."
  - tier: logging
    commitment_delta:
      surface: recorded-history
      delta: "structured event records (level, timestamp, context) emitted to a store an operator can read/search. The WHAT of past events is recoverable for the retention window."
    admin_tool: "log viewer / search; an alert surface over log events."
    enforcement:
      - "key operations emit a structured event; sensitive fields are redacted."
    tests:
      - "a diagnostic event is emitted with the expected fields; redaction holds."
    forecloses: "aggregate/trend questions (logs answer 'what', not 'how often' cheaply)."
    migration_cost_to_next: "medium — add aggregated metrics (counters/gauges/histograms)."
  - tier: metrics
    commitment_delta:
      surface: recorded-history
      delta: "aggregated time-series (counters, gauges, histograms) over the system's behavior; trends + rates + percentiles are answerable for the retention window."
    admin_tool: "metrics dashboard + alerting on thresholds."
    enforcement:
      - "key rates/latencies/saturation are instrumented; cardinality is bounded."
    tests:
      - "a metric advances on the measured event; an alert fires at threshold."
    forecloses: "per-request causal reconstruction (metrics aggregate away identity)."
    migration_cost_to_next: "high — propagate trace context across component boundaries."
  - tier: tracing
    commitment_delta:
      surface: recorded-history
      delta: "causally linked spans carrying a trace context across component/process boundaries; a single request's full path is reconstructable."
    admin_tool: "trace explorer (waterfall / span tree); sampling config."
    enforcement:
      - "trace context is propagated across every boundary; spans are linked parent→child."
    tests:
      - "a multi-component request produces a connected trace; context survives a process boundary."
    forecloses: "boundary-blind reasoning; every component must propagate context."
    migration_cost_to_next: "n/a — top of the ladder."
examples:
  - "The skill's own climb: the daemon publishes an operational observability surface — get_daemon_status + get_daemon_alerts (the AlertLog, #354, sqlite-durable), the SSE event stream that drives the live UI, and a /health endpoint distinct from request success (the #680 liveness fix turned on heartbeat-vs-EventSource-error distinction). Sits at logging+metrics (alerts + status are event records + light aggregates); not tracing — there is no cross-component span propagation, by current design. The recorded-history foreclosure bit live: liveness windows that weren't instrumented were undiagnosable until the surface was added (the a7/#680 SSE-signal diagnosis)."
```

## Selection procedure (§8)

Bidirectional check; chosen once. Recorded in the system architecture record; a
critical operation with no observability emission at the system's declared tier
is a systems-analysis D-finding.

## Composition (§5)

Observability is the operational sibling of the **audit log = principal ×
temporality × authorization** product — both record who/what/when, observability
for diagnosis, audit for accountability. Rule authored in 5A.3.
