# v1.9.5 stability report — per-node verdicts

**Date:** 2026-05-13
**Release:** v1.9.5
**Topology under test:** v1.9.3 (frozen 2026-05-09; 9 nodes)
**Corpus:** v1.9.4 combined 141 domains (50 hardened-adversarial +
91 soft v1.9.0).
**Predecessor reports:**
[v1.9.4](v1.9.4-calibration.md) (hardened-adversarial validation),
[v1.9.3](v1.9.3-calibration.md) (topology surgery),
[v1.9.0](v1.9-validation-summary.md) (initial calibration).

## Headline

v1.9.5 decides — does not ship — per-node stability criteria. The
`stability: stable | experimental | deprecated` schema field is held
for the first post-v2.0 patch that adds a node not immediately
qualifying as `stable` (per `docs/roadmap.md` §v1.9.5). At v2.0
release time the network ships with the disposition table below;
nodes are accounted for by name, not by per-node label.

| Verdict | Count | Nodes |
|---|---|---|
| **stable** | 7 | `m365_tenant`, `google_workspace_tenant`, `federated_identity`, `email_gateway_present`, `email_security_modern_provider`, `cdn_fronting`, `aws_hosting` |
| **not yet** | 2 | `okta_idp`, `email_security_policy_enforcing` |

Both `not yet` nodes carry explicit dispositions below. Neither
disposition is *Split* or *Remove*; both remain in the network as
v2.0 ships.

## Methodology

Each node is assessed against three criteria from
`docs/roadmap.md` §v1.9.5. `stable` requires **all three** to pass;
a node clearing two but not the third is `not yet`.

### Criterion (a) — evidence-response correctness

For every node:

1. **Bound binding raises posterior.** Each of the node's evidence
   bindings (slug or signal) raises the node's marginal posterior
   strictly above the no-evidence baseline by ≥ 0.01. For
   pure-propagation nodes (`email_security_modern_provider`), each
   parent's evidence binding raises the child via the CPT.
2. **Unrelated binding is inert.** A binding that is d-separated
   from the node under test (in the marginal — no other nodes
   observed) leaves the node's posterior at the no-evidence
   baseline within 1e-9.

**Implementation:** `tests/test_node_stability_criteria.py` —
parametrized over every node. The test failing is a regression
signal, not a v1.9.5-only check. A topology change that breaks
propagation for any node fails here before reaching a release tag.

### Criterion (b) — calibration in both regimes

Two binary checks against the deterministic pipeline as a proxy
label. No ground-truth probability claim is made; this measures
agreement between the Bayesian layer and the deterministic
pipeline, not real-world correctness.

* **(b1) Det-positive HIGH → Bayesian high.** When at least one of
  the node's evidence bindings fires at HIGH deterministic
  confidence (slug confidence ≥ 0.70, or any signal firing), the
  posterior must be > 0.5 and the credible interval must not span
  the full [0, 1]. Required: 100% of eligible observations.
* **(b2) Det-silent → interval widens.** When no evidence bindings
  fire, the layer must flag the observation `sparse=true` (its own
  signal that the interval was widened rather than collapsed on a
  confident point estimate). Required: 100% of eligible observations.

Brier score, log-score, and expected calibration error (ECE) are
recorded as diagnostics. Bad numbers flag a node for CPT / topology
re-examination but are not auto-failure criteria (per roadmap
§v1.9.5).

**Diagnostic thresholds (advisory only):** ECE ≤ 0.20; Brier ≤
0.15. Every node in this run lands well under both.

### Criterion (c) — independent firings

Number of distinct domains in the combined 141-domain corpus where
`evidence_used` for this node is non-empty. **Required: ≥ 10**.
Pure-propagation nodes (no direct evidence by design) are marked
`n/a`; stability gates on (a) and (b) only for those.

**No fast-tracking on numbers alone.** A node with excellent ECE
but only 4 firings does *not* pass — the threshold is *all three*.

## Per-node verdict table

| Node | (a) | (b1) | (b2) | (c) firings | Brier | Log-score | ECE | Verdict |
|---|---|---|---|---|---|---|---|---|
| `m365_tenant` | pass | 91/91 | 50/50 | 86 (pass) | 0.0033 | 0.057 | 0.055 | **stable** |
| `google_workspace_tenant` | pass | 45/45 | 94/94 | 47 (pass) | 0.0121 | 0.116 | 0.109 | **stable** |
| `federated_identity` | pass | 36/36 | 105/105 | 36 (pass) | 0.0046 | 0.068 | 0.066 | **stable** |
| `okta_idp` | pass | 7/7 | 134/134 | **7 (fail)** | 0.0223 | 0.156 | 0.144 | **not yet** |
| `email_gateway_present` | pass | 42/42 | 99/99 | 40 (pass) | 0.0084 | 0.096 | 0.092 | **stable** |
| `email_security_modern_provider` | pass | 0/0 | 141/141 | n/a (pure propagation) | — | — | — | **stable** |
| `email_security_policy_enforcing` | pass | **119/129** | 12/12 | 122 (pass) | 0.0346 | 0.152 | 0.128 | **not yet** |
| `cdn_fronting` | pass | 55/55 | 46/46 | 88 (pass) | 0.0008 | 0.027 | 0.026 | **stable** |
| `aws_hosting` | pass | 19/19 | 109/109 | 31 (pass) | 0.0046 | 0.063 | 0.061 | **stable** |

## Stable-verdict reads

Seven nodes clear all three criteria.

`m365_tenant`, `google_workspace_tenant`, `federated_identity`,
`email_gateway_present`, `cdn_fronting`, and `aws_hosting` are
evidence-driven roots (or, in `federated_identity`'s case, a
CPT-with-evidence node) with adequate corpus exposure and tight
calibration. Their (b1) and (b2) ratios are perfect, their ECE
values are well under 0.20, and their independent firing counts
range from 31 (`aws_hosting`) to 88 (`cdn_fronting`).

`email_security_modern_provider` is a pure-propagation node — no
direct evidence bindings by design. Criterion (c) is marked `n/a`
(structurally inapplicable); the (a) test exercises propagation
from each of its three parents (`m365_tenant`,
`google_workspace_tenant`, `email_gateway_present`) and confirms
the child's posterior moves as expected. (b2) is trivially 141/141:
without direct evidence, every observation is det-silent, and the
layer correctly flags every one as sparse.

## `not yet` dispositions

### `okta_idp` — **disposition: keep, expand corpus**

* **Why not yet:** 7 firings on the 141-domain combined corpus
  (criterion (c) requires ≥ 10).
* **Diagnostic numbers within thresholds:** Brier 0.0223, log 0.156,
  ECE 0.144 — all comfortably under the diagnostic thresholds, with
  (b1) 7/7 and (b2) 134/134 perfect.
* **Why the firing count is corpus-limited, not engine-limited:**
  Okta's public DNS fingerprint is thin. recon's deterministic
  pipeline identifies Okta via the `okta` slug, which fires on the
  Okta-specific verification TXT record or the
  `*.okta.com` / `*.oktapreview.com` chain motif. Many Okta-using
  organizations don't surface either publicly. The Bayesian layer
  correctly fires (b1=7/7) when Okta evidence is present; it
  correctly hedges (b2=134/134) when it isn't.
* **Action:** None in v1.9.x. v2.0 ships with `okta_idp` in the
  network; the corpus expansion that gets it to ≥ 10 firings is a
  validation-process item, not an engine item. Revisit at the
  first post-v2.0 calibration pass.

### `email_security_policy_enforcing` — **disposition: redefine in v1.9.6**

* **Why not yet:** (b1) is 119/129 = 92.2%. Ten observations are
  det-positive HIGH (signal firing) AND non-sparse AND posterior ≤
  0.5.
* **Specific failure pattern (all 10 cases identical):** evidence
  used is `signal:dkim_present` alone. With `prior = 0.25` and
  likelihood `[0.85, 0.30]`, the resulting posterior is
  $\frac{0.25 \times 0.85}{0.25 \times 0.85 + 0.75 \times 0.30}
   = \frac{0.2125}{0.4375} \approx 0.486$ — just below 0.5.
  The single weak signal does not push the posterior over the
  threshold, and the n_eff threshold for sparse is satisfied by
  the single binding, so the layer doesn't hedge either.
* **Why this is a CPT-change-discipline question, not a CPT tweak:**
  Per `docs/roadmap.md` §v1.9.6, corpus disagreement is the
  *signal* to ask the topology question, not the *fix*. The right
  question is which conceptual claim `policy_enforcing` is
  making. Three candidate topology / CPT changes, each with the
  concept-comment requirement v1.9.6 will enforce:
  1. **Raise the prior.** The current 0.25 reflects the "ground
     truth" base rate of fully-enforcing DMARC policy across all
     domains; the corpus is enriched for enterprise apex domains
     where the rate is higher (likely 0.35–0.45). Argument:
     priors should reflect the *target population* the node
     answers about, which is enterprise apex domains, not the
     internet at large.
  2. **Widen the sparse threshold.** The current sparse cutoff
     accepts one binding as enough n_eff. Argument: a single
     weak-likelihood signal does not give the layer enough
     evidence to make a confident claim either way; widening
     sparse to require ≥ 2 bindings would correctly hedge these
     domains.
  3. **Tighten the weak likelihoods.** `dkim_present` likelihood
     `[0.85, 0.30]` has the smallest spread in the network.
     Raising the lower end (e.g., to 0.20) would mean DKIM
     presence carries more weight; alternatively, lowering the
     upper end (e.g., to 0.75) would acknowledge that DKIM
     presence does not strongly imply full policy enforcement.
* **Action:** v1.9.6 (CPT-change discipline) selects one of the
  three options, documents the concept in `bayesian_network.yaml`
  as a YAML comment alongside the change, and re-runs this
  calibration to confirm (b1) reaches 129/129. The node stays in
  the network throughout — no Split or Remove.

## Cross-references

* **(a) test in code:** `tests/test_node_stability_criteria.py` —
  20 parametrized assertions covering bound-evidence sensitivity
  and unrelated-evidence inertia for every node (including the
  pure-propagation case via parent-evidence propagation), plus
  two sanity tests (root-baseline = prior, directory completeness).
* **(b) and (c) analyzer:** `validation/compute_node_stability.py`
  — anonymized aggregates only; reproducible from the gitignored
  NDJSON via the invocation in *Reproducibility* below.
* **v1.9.4 calibration report:**
  [`v1.9.4-calibration.md`](v1.9.4-calibration.md) — the corpus
  this run uses; the hardened-adversarial subset's sparse
  fingerprints are documented there.
* **v1.9.6 milestone:** `docs/roadmap.md` §v1.9.6 — the CPT-change
  discipline that the `policy_enforcing` disposition routes into.

## Reproducibility

```bash
# Re-run the analyzer against the v1.9.4 corpus output.
python validation/compute_node_stability.py \
    --hardened validation/corpus-private/v1.9.4-hardened/results.ndjson \
    --soft     validation/runs-private/v1.9.4-soft-rerun/results.ndjson \
    --json-out validation/v1.9.5-stability.json

# Criterion (a) is exercised by pytest.
pytest tests/test_node_stability_criteria.py -v
```

The corpus NDJSON files live under `validation/corpus-private/` and
`validation/runs-private/`, both gitignored. Inclusion criteria for
the hardened subset are documented at the top of the v1.9.4
calibration report's *Reproducibility* section.

## What this run does NOT validate

1. **Real-world calibration.** The (b) criterion uses the
   deterministic pipeline's verdicts as a proxy label, not ground
   truth. A node that agrees with the deterministic pipeline may
   still be wrong about the underlying claim. The discipline here
   is layer-vs-layer consistency, not external validation.
2. **Cross-vertical generalization.** The 141-domain combined
   corpus skews enterprise and hardened-enterprise. Consumer-
   facing, niche-SaaS, or higher-ed verticals may surface
   different stability behavior. See `docs/roadmap.md` post-v2.0
   backlog for the cross-vertical generalization study.
3. **Stability under topology change.** This run validates the
   *current* v1.9.3 topology. Any v1.9.x patch that adds, splits,
   or removes a node must re-run both the (a) test (auto, via
   pytest) and the (b)(c) analyzer.

## v2.0 implications

v2.0 ships without the `stability` schema field. The disposition
this report records is structural: 7 nodes stay, 2 nodes stay
with explicit follow-up patches scheduled. No node is removed at
v2.0; no `experimental` label survives.

The `stability` field itself enters the schema the first time a
post-v2.0 patch introduces a node that does not immediately
qualify as `stable` — i.e., when the field has something
meaningful to express. Until then, the absence of the field is
the strongest possible statement: every node in the shipped
network has been examined against these criteria and remains
because it earned it.
