# FEATURE.md — Content serialization: ONE shape, ONE deserializer

Verified against code 2026-07-29.

A `chat.message.content` block is the platform's most-round-tripped shape: written by the
provider adapters, stored as JSONB, rebuilt on every turn, re-sent to a provider, mirrored to
desktop, and rewritten by clients. **Every byte of it is provider-critical.** A key lost anywhere
in that loop does not fail where it was lost — it fails several turns later as a provider 400 that
kills every remaining turn of the conversation.

---

## The rule

**A persisted shape has ONE deserializer.** For content blocks that is
[`reconstruct_content`](unified_content.py) (plus [`reconstruct_media_content`](media_config.py)
for `type="media"` and `reconstruct_structured_input` for structured inputs).

- **`UnifiedMessage.parse_content` normalizes wire aliases, then DELEGATES.** It may map
  `input_text`→`text`, `image_url`→`url`, `function_call`→`tool_call`. It must **never** rebuild a
  canonical block itself.
- **NEVER rebuild a persisted shape with a field-name filter.** `_filter_kwargs` keeps only keys
  whose name matches a dataclass field, so every storage key that is spelled differently
  (`call_id` → `id`) or needs decoding (`<key>__b64` → bytes) vanishes **silently** — no
  exception, no log.
- **Storage key ≠ field name is normal, and it is exactly what gets lost.** `to_storage_dict()`
  emits `call_id` for `ToolCallContent.id`, base64-encodes bytes under `<key>__b64`, and lifts
  citations to the top level. The deserializer owns every one of those translations.
- **Normalization must be IDEMPOTENT.** `normalize(normalize(x)) == normalize(x)`. A normalizer
  that fails to recognize its own output re-wraps it and drops fields on every read-write cycle.
- **🚨 THE SAME RULE BINDS THE WRITE SIDE: a SERIALIZER is never a display form.** `to_dict()` and
  `to_storage_dict()` are both rebuilt-from; `__repr__` is the only place allowed to redact.
  Binary provider-continuity material has exactly ONE encoding —
  [`encode_binary_metadata`](config_utils.py) (base64 under `<key>__b64`), decoded by
  `decode_binary_metadata` inside `reconstruct_content`. No serializer may emit the
  `<bytes length=N>` / `<str length=N>` placeholders (a parity test asserts it), and
  `MessageList.to_dict_list()` delegates to each block's `to_dict()` rather than dumping
  `content.__dict__` — that dump was a fourth serializer of the same shape.

## Why — three incidents in three days, one mechanism

| Date | Lost | Blast radius |
|---|---|---|
| 2026-07-16 | `citations` (top-level storage key, not a dataclass field) | Every reparse of a stored message erased its citations |
| 2026-07-18 | `call_id` (storage spelling of `ToolCallContent.id`) | `tool_use.id=""` → Anthropic 400 on **every later turn**; paired `tool_result` dropped as an orphan; one live conversation bricked |
| 2026-07-18 | `google_thought_signature` (`__b64` metadata, left as a base64 str) | `to_google()` silently omitted `thoughtSignature` → Gemini 3 400 on replay |
| 2026-08-16 | `google_thought_signature` again — this time on the WRITE side | `to_dict_list()`'s `__dict__` dump + the `__repr__` redactor inside `to_dict()` wrote `<bytes len=N>` into `chat.request_snapshot.unified_payload`, so **every recorded Gemini call was un-replayable** (151 SDK validation errors before the request left the process) and a whole provider's history was unusable as evidence. Guard extended to the serializers; see [`tests/test_google_wire_replay_payload.py`](../../tests/test_google_wire_replay_payload.py) |

All three: `parse_content` was a **second, independent implementation** of a shape
`reconstruct_content` already owned, and the two drifted. The first two were patched pointwise;
the third was found only by building the parity guard below. Pointwise patch #4 is not the fix —
delegation is.

A fourth of the same family lived in [`citations.py`](citations.py): `is_normalized_citation`
required `raw`, which is optional-with-default, so a canonical citation carrying no provider
payload was re-normalized down the "unknown legacy shape" branch and rebuilt from only
`cited_text`/`title`/`url` — silently dropping `source_start`/`source_end`/`page` on every cycle.

## The guard — [`tests/test_content_deserializer_parity.py`](../../tests/test_content_deserializer_parity.py)

Two properties, asserted for every content class × every key needing translation:

1. **PARITY** — `reconstruct_content(stored)` and `parse_content([stored])[0]` produce a
   semantically identical object. A divergence is the exact shape of all three incidents.
2. **IDEMPOTENCE** — `store → rebuild → store` is stable on both paths, so a row never degrades
   by being read and written.

Plus: `__b64` metadata comes back as **bytes** on both paths (a base64 str is invisible to an
`in metadata` check, so the provider payload silently omits it), the tool-pairing join key
survives every spelling, and a malformed cosmetic field (`output_chars: "abc"`) never raises.

Media blocks follow the same rule through `reconstruct_media_content`. A definitive MIME family
reconciles a stale `image|audio|video|document` label before construction. This is a loud
recovery: it logs the claimed and corrected kinds plus the locator, then persists the truthful
kind on the next storage round trip. This prevents a URL-only `image/jpeg` attachment labeled
`document` from entering document fallback and being dropped before provider dispatch.

**Adding a content class or a storage key → add a sample to `SAMPLES`.** If parity fails, fix
`reconstruct_content`; never special-case `parse_content` again.

## Related invariants

- **Message-part Pydantic models are the persisted contract.**
  [`db/message_parts.py`](../db/message_parts.py) owns every
  `chat.message.content[]` variant, including id-backed agent, project, app,
  transcript, workbook, and rich-document inputs. `validate_message_content`
  rejects unregistered discriminators before a DB write.
- **`user_input` is typed at the HTTP boundary.** `UserInputPart` validates
  against the same variants, normalizes legacy media aliases, and returns plain
  dictionaries for the existing assembly pipeline. Never widen an execution
  request back to `list[dict[str, Any]]`; OpenAPI must expose the union.
- **Generated TypeScript discriminators are required.** The stream generator
  forces `type`/`kind` to be non-optional and derives the runtime
  `parseMessageContent` validator from the same Pydantic JSON Schema. A
  cast-only or hand-mirrored database parser is forbidden.
- **Resolvable payloads are non-empty and exact.** Web URLs, bookmark identity
  fields, entity-id arrays, and media locators require non-empty strings;
  structured arrays require at least one member. `input_data.refs` is the exact
  discriminated `DataRef` union. Notes/tasks accept an exact live-reference or
  by-value snapshot union; workbook/document accept only live references.
- **JSON Schema constraints survive generation.** `allOf` locator requirements,
  `minLength`/`minItems`, nested `additionalProperties`, and exact union members
  are preserved in static TypeScript and the generated runtime validator.
- **Every persisted id-backed attachment has a model-context implementation.**
  `input_agent`, `input_project`, `input_agent_app`, `input_transcript`,
  `input_transcript_session`, `input_workbook`, and `input_document` reconstruct
  through `STRUCTURED_INPUT_TYPE_MAP` and emit a canonical `matrx` reference
  fence. The host resolves that fence through its RLS-scoped reference registry
  for the provider send-clone. Workbook/document also retain their dedicated
  on-demand content-tool instruction; their reference resolves live metadata.
- **The tool-pairing graph is server-owned and immutable after write.** Guarded at the DB
  (`chat.message_tool_graph_write_guard`, migrations 0151/0207), at the REST edge
  (`PATCH /conversations/{id}/messages/{id}` → 409 `tool_call_graph_change_forbidden`), and at the
  provider boundary (`MessageList.sanitize` — Pass 1.5 repairs an id-less `tool_use` by adopting
  the unique adjacent `tool_result`'s id, else drops it; an empty id must never reach a provider).
- 🚨 **`tool_use` blocks are the LAST assistant-side content before their results — a
  producer that emits anything in between ERASES the tool call.** Every provider requires a
  `tool_result` in the message immediately following its `tool_use`, so `MessageList.sanitize`
  drops a `tool_use` that lost adjacency, then its now-orphaned `tool_result`, then the
  emptied `role='tool'` message. When nothing duplicates the id, that drop is pure DATA LOSS:
  the call vanishes from the model's own history AND is never written to `chat.message`, so
  the user watches the tool card stream and loses it on reload (`chat.tool_call` still has the
  row, with `message_id` NULL). A response translator that splits ONE provider turn into
  several `UnifiedMessage`s **must put the tool calls last**, in their own `assistant`
  message — see the ordering note in
  `providers/openai/translator.py::OpenAITranslator._build_unified_messages`, pinned by
  [`tests/test_openai_tool_call_message_adjacency.py`](../../tests/test_openai_tool_call_message_adjacency.py).
  The guard now classifies each drop (`reason: "separated"` vs `"duplicated"`) so this shape
  can never again be reported as somebody else's duplicate bug.
- **`is_visible_to_model=false` rows never reach a provider** — see
  [STATUS_AND_ERROR_FIELDS.md](../../../../docs/persistence/STATUS_AND_ERROR_FIELDS.md).

## LLM parameter normalization

`LLMParams` is the typed request boundary and `UnifiedConfig.__post_init__` is
the persisted/runtime boundary. Both route `max_output_tokens` through
`normalize_max_output_tokens`: a numeric value with a fractional part is
truncated to an integer before any provider translator sees it. Invalid values
remain invalid and are rejected by Pydantic; this normalization is not a
fallback for malformed input.

Provider-native state that must survive a later turn is a typed LLM parameter,
not ad-hoc metadata. Google Interactions uses `previous_interaction_id` for
conversational video editing and `task` for the documented video task enum;
both live on `LLMParams` and `UnifiedConfig`, round-trip through conversation
storage, and are included in the generated frontend key set.

## Change Log

- **2026-08-11** — Stopped the OpenAI translator erasing a tool call: it grouped a turn's
  `function_call` blocks with the reasoning and emitted the assistant TEXT message after
  them, so a text message sat between a `tool_use` and its `tool_result` and
  `MessageList.sanitize` deleted the pair. Tool calls now come last in their own `assistant`
  message; the guard classifies `separated` vs `duplicated` drops instead of calling both
  duplicates (251 mislabelled alarms in the preceding 30 days).
- **2026-08-11** — Closed the persisted-to-provider gap for all seven id-backed
  attachment variants by adding reconstruction and canonical reference-fence
  resolution; workbook/document now use the same reference path for metadata.
- **2026-08-11** — Made persisted and outbound message parts one validated
  Pydantic/OpenAPI contract; added all id-backed attachment variants and a
  generated runtime parser with required discriminators.
- **2026-08-11** — Tightened attachment payloads so empty URLs, locators,
  bookmark identities, arrays, and opaque snapshots fail at ingress; generated
  TypeScript/runtime parsing now preserves the same constraints and snapshot
  extension fields.
- **2026-08-08** — Classified authored/effective tool declarations and host-policy
  transition markers as internal `UnifiedConfig` state in the LLMParams drift gate.
  They remain durable across turns but cannot be forged as caller overrides.
- **2026-07-29** — Routed wire document/file aliases through `reconstruct_media_content` and
  added MIME-based kind reconciliation for image/audio/video/document blocks. Uploaded images
  mislabeled as documents now become `ImageContent` while preserving `file_id` or URL; recovery
  fires a loud diagnostic and regression tests cover wire, stored, and non-image media shapes.
- **2026-07-27** — Normalized fractional `max_output_tokens` values at both the
  typed request boundary and `UnifiedConfig` hydration/direct construction so
  persisted decimals can never reach a provider.
- **2026-07-18** — Created. Collapsed `parse_content`'s canonical branches (`text`, `thinking`,
  `tool_call`, `tool_result`) into `reconstruct_content`; made `reconstruct_content` preserve
  client `metadata` on text blocks and fail-soft on a bad `output_chars`; made
  `is_normalized_citation` idempotent; added the parity guard.
