# v1.9.9 — synthetic corpus run

**Release:** v1.9.9
**Date:** 2026-05-14
**Corpus:** `validation/synthetic_corpus/results.json` (19 fixtures,
Microsoft-fictional brands).
**Aggregator:** `validation/corpus_aggregator.py`.

## Why a synthetic corpus

The v1.9.9 test surfaces are exercised against unit-test fixtures and
the v1.9.2 agentic-UX fixtures (`contoso-dense.json`,
`hardened-sparse.json`). Both fixture sets are minimal by design:
they pin one or two trigger conditions each. They do not exercise
the v1.9.9 surfaces across the *distribution* of realistic public
stack shapes an operator might query.

This corpus closes that gap with 19 hand-curated TenantInfo fixtures
modelled after public stack shapes observed in defensive review:

- Mid-size enterprise on Microsoft 365 + Okta + AWS.
- Engineering-heavy organizations on Google Workspace + AWS-native.
- SaaS-heavy multi-cloud with three vendor families.
- Hardened-target minimal-DNS posture (sparse public footprint).
- GCP-native startup, Azure-native enterprise, single-cloud patterns.
- Hybrid M365 + Google Workspace co-existence.
- Modern Jamstack (Vercel + Cloudflare).
- Healthcare-vertical heavy-security posture.
- Public-sector hardened posture (.gov apex).
- Media publisher with heavy CDN footprint.
- Fintech with federated identity and many security tools.
- Higher-ed with LMS + Zoom + GWS.
- Heroku-hosted legacy application.
- SaaS-only with no cloud infrastructure slugs.
- Pure-AWS apex with multiple AWS-family slugs.
- Minimal degraded-lookup result.

All apex names use the Microsoft fictional brand convention (Contoso,
Northwind, Fabrikam, Adatum, Tailspin, Wingtip, Litware, Trey
Research). No real organizations are depicted. The corpus and the
aggregator script are both committed to the public repo and the
output is reproducible.

## How to reproduce

```
python validation/synthetic_corpus/generator.py
python validation/corpus_aggregator.py \
    validation/synthetic_corpus/results.json \
    --output validation/synthetic_corpus/aggregate.json
```

## Per-fixture intended behaviour and observed firings

The aggregator emits two firing counts per surface: an *estimator-based*
count (fast, uses a slug-only lower bound on the categorized service
count) and a *render-based* count (slow, runs the actual renderer).
The render-based count is authoritative. The estimator over-fires the
ceiling vs reality because pass-2 prefix classification raises the
categorized count for some fixtures above the slug-only floor.

| Fixture | Intended multi-cloud | Intended ceiling | Notes |
|---|---|---|---|
| m365_okta_enterprise | Fire (AWS + Cloudflare) | Suppress (rich-stack) | Trigger discipline on rich-stack |
| google_workspace_aws_native | Suppress (AWS only) | Fire (sparse-ish) | Single-cloud + multi-domain |
| multi_cloud_saas_heavy | Fire (AWS + Cloudflare + GCP + Fastly) | Suppress (5 surface attribs) | Surface-attribs cutoff |
| hardened_minimal_dns | Suppress (Cloudflare only) | Fire (minimal DNS) | Canonical hardened case |
| gcp_native_startup | Suppress (GCP only) | Fire-ish (small but multi-domain) | Single-cloud sparse |
| azure_native_enterprise | Suppress (Azure only) | Fire-ish | Single-cloud sparse |
| small_single_domain_org | Suppress | Suppress (domain_count == 1) | Genuinely-small org gate |
| saas_only_no_cloud | Suppress (no cloud slugs) | Fire | SaaS-only sparse |
| hybrid_dual_email | Fire (AWS + Cloudflare) | Fire (categorized < 5) | Multi-vendor sparse |
| cdn_fronted_minimal | Suppress (Cloudflare only) | Suppress (domain_count == 2) | Multi-domain gate |
| vercel_jamstack | Fire (Vercel + Cloudflare) | Variable | Modern stack |
| healthcare_compliance | Fire (AWS + Cloudflare) | Suppress (rich) | Vertical posture |
| public_sector_hardened | Suppress | Fire | .gov hardened |
| media_publisher_heavy_cdn | Fire (3 vendors) | Suppress (9 attribs) | Heavy-surface case |
| fintech_high_security | Fire (AWS + Cloudflare) | Suppress (rich) | Fintech posture |
| education_lms_heavy | Suppress (AWS only — Canvas excluded) | Fire | SaaS-not-cloud discipline |
| heroku_legacy_app | Fire (Heroku + Cloudflare) | Fire | Heroku rollup verification |
| empty_minimal | Suppress | Suppress (services empty) | Degraded-lookup case |
| two_aws_slugs_one_vendor | Suppress (AWS family) | Fire | Canonicalization discipline |

## Aggregate result

```json
{
  "corpus_size": 19,
  "counted": 19,
  "multi_cloud": {
    "rendered_fired": 8,
    "rendered_fired_share": 0.421
  },
  "ceiling": {
    "rendered_fired": 11,
    "rendered_fired_share": 0.579
  }
}
```

**Multi-cloud rollup**: 8 of 19 (42.1%). This matches the intended
design — fires on every multi-vendor case (Contoso, Fabrikam,
Wingtip Media, Litware Capital, Adatum Health, Fabrikam Marketing,
Contoso Mergers, Contoso Legacy Apps) and suppresses on every
single-cloud case (Northwind AWS-native, Adatum hardened, Tailspin
GCP, Wingtip Azure, etc.) plus the SaaS-only and pure-AWS-family
cases. Canonicalization (AWS Route 53 + CloudFront + S3 + EC2 = one
AWS vote) is verified across `google_workspace_aws_native` and
`two_aws_slugs_one_vendor` — neither fires despite 3+ AWS-family
slugs each.

**Passive-DNS ceiling**: 11 of 19 (57.9%). This is higher than a
typical defensive-review corpus would produce because the synthetic
corpus is intentionally weighted toward both surfaces — many
fixtures are constructed to exercise the ceiling. On a balanced
operational corpus, the ceiling firing share would be substantially
lower; the gitignored private corpus is the appropriate measurement
for that question.

The fixture pairs that anchor the trigger discipline:

- **Fires correctly on hardened minimal DNS**: `hardened_minimal_dns`
  (8 tenant domains, 1 service, 0 attribs) and `public_sector_hardened`
  (.gov apex with 2 services, 0 attribs).
- **Suppresses correctly on rich-stack**: `m365_okta_enterprise`
  (7 services, 4 attribs), `multi_cloud_saas_heavy`, `healthcare_compliance`,
  `fintech_high_security`, `media_publisher_heavy_cdn`.
- **Suppresses correctly on genuinely-small org**: `small_single_domain_org`
  (`domain_count == 1`).
- **Suppresses correctly on empty services**: `empty_minimal`
  (no `info.services`).
- **Suppresses correctly on multi-domain-with-many-attribs**:
  `cdn_fronted_minimal` (`domain_count == 2`, just below the
  multi-domain gate).

## Validation conclusion

On this synthetic corpus, both v1.9.9 surfaces behave as designed:

- Multi-cloud rollup canonicalization holds across AWS-family,
  Azure-family, GCP-family (Firebase included), and standalone
  vendors (Cloudflare, Fastly, Heroku, Vercel). SaaS slugs that are
  cloud-categorized but excluded (Canvas LMS, etc.) do not contribute
  to vendor counts.
- Ceiling trigger fires only on the conservative trigger envelope
  (multi-domain + sparse-services + sparse-attribs + non-empty
  services). Single-domain orgs and rich-stack orgs are correctly
  suppressed.

The remaining honest gap (acknowledged in
`validation/v1.9.9-detection-gap-ux.md` test-quality manifesto)
is that this corpus is synthetic. The aggregator is designed to run
against the gitignored private corpus next; both runs will share the
same output shape so the v1.9.9 surfaces' behaviour can be compared
between synthetic and real-corpus distributions before v2.0 lock.

## Next steps

1. Run the same aggregator against the private v1.9.4 hardened
   corpus when next opened (gitignored; output aggregate-only,
   safe to commit alongside this memo).
2. If the private-corpus ceiling firing rate exceeds the synthetic
   rate by a wide margin, the conservative trigger may still be
   over-firing on real targets; revisit the categorized-count
   threshold.
3. If the private-corpus multi-cloud firing rate is *under* the
   synthetic rate, the canonicalization map may have a coverage
   gap on real-world slug shapes; revisit the rollup map.
