# Memory Persistence Plan — Cohort Lifetime Across Axiom Versions

**Status:** Active draft (2026-04-27)
**Owner:** Ben Booth
**Forcing context:** The Prague cohort is the first live cohort. It must continue to function, with its data readable and its agents alive, across every Axiom (and Vega) release for years. Future cohorts adopt whatever shape Prague establishes — so getting persistence right matters now, not later.
**Related:** `spec-memory.md`, ADR-033, `prd-memory.md`, `working/memory-roadmap.md`.

---

## 1. Why this plan exists

Prague is a precedent. The data shape we ship in May 2026 is the data shape every Axiom upgrade through 2031+ has to read. If we change the `MemoryFragment` schema in 0.13 in a way that doesn't migrate, we either break Prague's data or fragment future cohorts. Neither is acceptable.

Three failure modes this plan rules out:

- **Silent schema break** — a field rename in 0.12 makes 0.11-written fragments un-decodable. Cohort dies on upgrade.
- **Lossy migration** — a 0.13 release "migrates" by dropping fields it didn't expect. Audit trail compromised; provenance chains broken.
- **Cohort-specific fork** — Prague stays on 0.11 because upgrade is too risky. Federation eventually fragments by version.

We avoid all three by making schema versioning a first-class data field, requiring forward-compat decode by default, and gating every release on a roundtrip test against pinned cohort fixtures.

## 2. Core invariants (normative for every Axiom release)

Pin these as compliance gates per `working/memory-benchmarks.md §2`:

### I-P1 — Schema version is on every fragment

Every `MemoryFragment` carries an explicit `schema_version: int` field, set at write time, immutable. The current spec is `1`. Every read path consults this field to choose its decoder.

### I-P2 — Forward-compat decode by default

Any Axiom version 0.11+ MUST decode every prior version's fragments without data loss. New fields default; renamed fields fall back to old names; removed fields generate audit-log entries (never silent drops). Decode that cannot complete cleanly fails closed with a structured error — never silently mangles.

### I-P3 — Backward-compat write per major version

A fragment written by Axiom 0.13 SHOULD be readable by 0.11 if no 0.13-only fields are populated. This is the federation requirement: if Prague's coordinator runs 0.11 while a peer node runs 0.13, the peer's fragments project across without forcing the coordinator to upgrade.

### I-P4 — Pinned cohort fixtures gate every release

Every release runs the compliance suite against a frozen Prague-shaped fixture cohort: ~100 fragments, mixed cognitive types, known-good projections. Failure = release blocker.

### I-P5 — Migration helpers, never silent rewrites

If a major schema change requires data movement (e.g., Stage 4 blob extraction moves payload bytes from `content` to `blob_refs`), it ships as an explicit `axi memory migrate` step. Old data is preserved (tombstoned, not deleted); new fragments reference the migrated shape. Migration is replayable + auditable.

## 3. The schema_version field

Add to `MemoryFragment` per spec-memory §3.1:

```python
@dataclass(frozen=True)
class MemoryFragment:
    # Existing fields ...
    schema_version: int = 1   # current spec version; never written-by-decode
```

**Decode rules per `fragment_from_dict`:**

```python
def fragment_from_dict(data: dict) -> MemoryFragment:
    version = data.get("schema_version", 1)
    if version > _CURRENT_SCHEMA_VERSION:
        raise UnsupportedSchemaError(
            f"fragment schema_version={version} > "
            f"{_CURRENT_SCHEMA_VERSION}; upgrade Axiom to read this fragment"
        )
    decoder = _DECODERS_BY_VERSION[version]
    return decoder(data)
```

`_DECODERS_BY_VERSION[v]` is a registered function that knows how to decode version `v`. New versions register a new decoder + leave the prior decoders intact. Decoders for old versions never get removed; they get pinned.

## 4. Versioning policy — when does the version bump?

| Change kind | Version impact |
|---|---|
| Add an optional field with safe default | No bump. Old readers ignore the field; new readers read it; new writers populate it. |
| Add a required field | **Bump.** Old readers can't honor the requirement; need explicit decoder. |
| Rename a field | **Bump.** New decoder maps either name; old decoder reads only the old name. |
| Remove a field | **Bump.** Old data preserved by old decoder; new decoder ignores. |
| Change semantics of an existing field's value space | **Bump.** Decoder routes by version. |
| Add a new cognitive type | No bump. Type is a string; old decoders pass it through; if they need to project they can fall back to a sensible default. |
| Add a new visibility horizon level | **Bump.** Default-deny composition needs to know the new level exists. |
| Add a new classification level | **Bump.** Same reason. |

The bumping decision is a release-note checklist item, not a judgment call. PRs that change `MemoryFragment` MUST cite the policy row + propose a version bump if applicable.

## 5. Migration helpers

When a release requires data movement (not just decode shape), ship a one-shot migration:

```bash
axi memory migrate <scope_id> --from <ver> --to <ver> [--dry-run]
```

Implementation pattern (v0):

- Read every fragment in the scope at version `<from>`.
- Apply the migration function: `migrate_v1_to_v2(frag) -> frag`.
- Write the migrated fragment as a new event with `schema_version = <to>`.
- Tombstone the original fragment with `reason="migrated_to_v<to>"`.
- Both lines persist on disk.

This pattern preserves the I2 append-only invariant (no UPDATE on existing fragments) and keeps the migration replayable (re-running is a no-op because the original is tombstoned + the new fragment exists).

**Examples this plan anticipates:**

- **Stage 4 blob extraction (v1 → v2):** Large `content` payloads move to `blob_refs`; `content` shrinks to metadata only. Migration: hash the bytes, put into BlobStore, replace content payload with the blob_ref.
- **Top-level scope field (v1 → v2):** `MemoryFragment.scope: str` becomes a top-level field instead of buried in `content["scope"]`. Migration: copy from content to top-level; old decoder falls back to content lookup.
- **Federation gateway promotion (v1 → v2):** `MemoryFragment.signed_by_gateway: list[Principal]` records the gateway chain. Migration: empty list for pre-gateway fragments.

Each of these is opt-in per scope. Edge / Workstation profiles can defer until they need the new feature; Server / Platform profiles migrate at release rollout.

## 6. Cohort-fixture compliance — pinning Prague

Add to `tests/memory/fixtures/cohort_prague_v1/` a frozen dump of:

- Cohort registry entry (cohort_id, members, coordinator pubkey)
- ~100 MemoryFragment artifacts spanning every cognitive type Prague writes
- Concept graph snapshot (post-Stage 2)
- Pinned projection outputs for known queries

Add to `tests/memory/test_persistence_compliance.py`:

```python
@pytest.mark.memory_compliance
def test_prague_v1_fixture_reads_under_current_axiom():
    """Every Axiom release MUST be able to read the pinned Prague v1 cohort
    fixture without data loss. This is the cohort-lifetime guarantee
    operationalized as a release-blocker test."""
    fixture = load_cohort_fixture("cohort_prague_v1")
    for raw in fixture.fragment_jsonl:
        frag = fragment_from_dict(raw)
        # Round-trip stable.
        assert frag.to_dict() == raw or _is_safe_field_addition(frag.to_dict(), raw)

@pytest.mark.memory_compliance
def test_prague_v1_projections_match_pinned():
    """Projections over the fixture cohort MUST match the pinned outputs
    byte-for-byte (replay determinism, I14)."""
    fixture = load_cohort_fixture("cohort_prague_v1")
    for query, expected in fixture.pinned_projections:
        actual = run_projection(fixture, query)
        assert actual == expected
```

These two tests gate every release. Any change that breaks them must either (a) ship a migration helper that produces a `_v2` fixture or (b) be reverted.

## 7. Federation-version compatibility

Cross-node federation under Stage 5 introduces a second compatibility surface: peer A on Axiom 0.11 talking to peer B on Axiom 0.13. Rules:

- **Outbound projections carry `axiom_version` + `schema_version` headers.** Peer can decide to accept based on its compatibility window.
- **Default acceptance window: ±1 major version.** A 0.11 peer accepts 0.10–0.12 projections. Wider windows are explicit policy.
- **Schema-version mismatch is rejected at gateway**, not silently degraded. Peer logs the rejection; operator sees it in audit.

This means Prague 0.11 can accept federated content from a 0.12 partner cohort transparently. A 0.15 partner is rejected — partner needs to project with `target_schema_version=0.11` (handled by the gateway re-projection function) or the operator widens the acceptance window.

## 8. What lands and when

This plan is documentation today. The implementation is staged:

| When | What |
|---|---|
| Now (this commit) | Plan + invariants documented |
| Stage-4-prep (next 1-2 weeks) | `schema_version: int = 1` field added to `MemoryFragment`; `_DECODERS_BY_VERSION` registry; round-trip tests |
| Stage 4 ship (Stage 4 in ADR-033) | First real version bump (v1 → v2 for blob extraction); migration helper; cohort fixture v1 → v2 path |
| Stage 5a ship (federation gateway) | `axiom_version` + `schema_version` headers on outbound projections; acceptance-window policy in `TrustProfile` |
| Every release thereafter | `pytest -m memory_compliance` gate + cohort fixture round-trip on every PR |

## 9. Open items

- **Cohort fixture format.** JSONL of fragments + JSON of projections is the strawman; format may need binary-stable canonicalization for byte-equivalent comparison across Python versions. Decide before fixture pinning.
- **Wide-version-window federation.** ±1 major as default; do we publish a "Prague is supported through Axiom 1.0" commitment publicly? Decide with the federation gateway design review.
- **Ownership of migration helpers.** Memory team owns the protocol; extensions own their per-fragment-type migrations. Spec out the per-extension migration responsibility in spec-memory follow-up.

## 10. The bottom line

If Prague's first fragment lives forever, this plan is what makes that possible. The implementation is small (one field, a decoder registry, two tests, one migration command) but the discipline matters: every PR that changes `MemoryFragment` consults the §4 policy table; every release runs the §6 fixture suite; every federation handshake checks §7 versions.

The cost of being careful here is small. The cost of skipping it is a cohort that dies the first time we change the data model — and that's the precedent that future cohorts notice.