# Context Assembly V2

**Status:** Proposed
**Owner:** context pipeline (`context_zones.py`, `voice_context.py`, `scripts/prompt_builder.py`, `telegram_listener.build_system_prompt`)
**Flag:** `CONTEXT_ASSEMBLY_V2` (default off; enable per-session for A/B)
**Supersedes:** current fixed-order assembly in `telegram_listener.build_system_prompt`

**Related:** [`DYNAMIC-CONTEXT-LAYERS.md`](./DYNAMIC-CONTEXT-LAYERS.md) · [`EMOTIONAL-DIRECTIVE-OVERHAUL.md`](./EMOTIONAL-DIRECTIVE-OVERHAUL.md) · [`TOKEN-LIMITS.md`](./TOKEN-LIMITS.md)

---

## 1. Motivation

Trace analysis of production turns (traces 1 and 3, 2026-04-12) shows three structural problems in V1 assembly:

1. **Turn-live steering sits in the middle trough.** `session_thread_grounding`, `conversation_compass`, and `user_need_directives` land in the 6k–12k character region of a ~25k-char system prompt — the zone long-context models systematically under-attend (lost-in-the-middle).
2. **Conflicting tone authorities stack unresolved.** `behavioral_cues`, the `mind_state` preamble, and `output_channel_contract` each shape voice. LLM conservative bias selects the most cautionary signal; warmth is lost. Pre-resolution is tracked in `EMOTIONAL-DIRECTIVE-OVERHAUL.md`; V2 consumes its output.
3. **Recency weight is spent on static content.** The channel contract correctly anchors the tail for tool-use fidelity, but no turn-specific content shares that high-attention region.

V2 keeps what works (channel contract tail anchor, zone concept, all existing stores) and restructures the middle to eliminate the trough for turn-live content.

## 2. Non-goals

- No new persistence stores. No new LLM calls. No schema changes.
- No change to history replay structure or user-turn construction.
- No rewrite of identity prose or operator config.
- Not a fix for fact deduplication in `learned_about_user` (tracked separately; V2 is neutral to it).
- Not a replacement for the emotional directive overhaul; V2 depends on its composite resolver.

## 3. Design: two-tail-anchor model

System prompt is assembled into four bands:

```
┌─────────────────────────────────────────────────────────┐
│ BAND 1 — TOP ANCHOR (static scaffold + primacy tone)      │
│   identity_shell                                        │
│   [when FUROSHIKI_DIRECTIVE_FIRST=1, default:]          │
│     behavioral_directives OR composite_voice_directive  │
│     live_time_anchor (NOW)                              │
│     narrator:time_and_emotions (state lines, if any)    │
│   addressing_context, user_profile (mini dossier),      │
│   inner_monologue_excerpt, plugin_context               │
│   [when DIRECTIVE_FIRST=0: time/emotions stay Band 3]   │
├─────────────────────────────────────────────────────────┤
│ BAND 2 — MID (slow-changing reference context)          │
│   learned_about_user, narrator:memories,                │
│   structured:user_anticipations,                        │
│   structured:emotional_weights, narrator:design_meta    │
│   last_conversation_recap (when injected)               │
├─────────────────────────────────────────────────────────┤
│ BAND 3 — TURN-LIVE TAIL (bounded ≤ 600 tokens)          │
│   [when DIRECTIVE_FIRST=0: composite + time/emotions   │
│    first; when DIRECTIVE_FIRST=1: those live in Band 1] │
│   operator_corrections, narrator:tasks, proactive_context │
│   session_thread_grounding, user_need_* ,               │
│   conversation_compass, thread_continuity_priority,     │
│   new_session_note, narrator:health                     │
├─────────────────────────────────────────────────────────┤
│ BAND 4 — STATIC TAIL ANCHOR                             │
│   output_channel_contract                               │
└─────────────────────────────────────────────────────────┘
[USER] <actual message>
```

Assembly code: `scripts/prompt_builder.py` (`assemble_v2_system_prompt`, `build_band1`–`build_band3`). See **§20** for flags and trace `band` tagging.

**Rationale.** Attention decay on long contexts is U-shaped. Band 1 and Band 4 claim head and tail for content that must survive decay (identity; tool rules). Band 3 is the high-recency slot adjacent to the channel contract for turn-specific steering that is not promoted to Band 1; with **`FUROSHIKI_DIRECTIVE_FIRST=1`**, the composite tone block and live state lines also claim early head attention (see Band 1 bullets).

## 4. Band specifications

### Band 1 — Top Anchor (target ~1,200 tok)

Persistent "who/when/with whom" frame. Primacy-weighted.
- `inner_monologue_excerpt` promoted from zone 2 → Band 1 because it frames stance, not recall.
- `plugin_context` promoted for the same reason (declares pre-surfaced capability state).
- **`FUROSHIKI_DIRECTIVE_FIRST=1` (default):** `behavioral_directives` (or `composite_voice_directive` label when the flag is off but content is still the same block) and `narrator:time_and_emotions` are inserted **immediately after** `identity_shell` so tone and live scalars anchor before the user dossier. Set **`FUROSHIKI_DIRECTIVE_FIRST=0`** to keep tone block in Band 3 only (legacy V2 ordering).

### Band 2 — Mid (hard cap 1,500 tok)

Retrievable reference material. Weakest attention; content here is "available" not "emphasized."
- **No behavioral rules.** Any "must/never" content here will be ignored; it belongs in Band 3.
- Structured blocks (`structured:*`) gain a one-line framing header so they have a read-side in the prompt. Without it they fail the behavioral-path rule.
- Overflow policy: drop lowest-similarity ChromaDB hits first, then oldest structured entries.

### Band 3 — Turn-Live Tail (hard cap 600 tok)

Everything needed to steer *this turn* specifically. Recency-weighted just above the tool-rules anchor. Order is load-bearing (see §10 risks):

1. `composite_voice_directive` / `behavioral_directives` — single pre-resolved tone block from `voice_context.build_behavioral_cues()`. **When `FUROSHIKI_DIRECTIVE_FIRST=1`, this slot is emitted in Band 1 instead** (Band 3 starts at `operator_corrections`). Target ≤ 300 tok after composite resolver work.
1.5. `operator_corrections` — structured operator feedback from `self_observations` (`source=operator`), when present. Injected immediately after the composite directive so authoritative corrections survive composite truncation before lower-priority slots. Listed last in `V2_BAND3_DROP_ORDER` in `scripts/context_zones.py` so overflow drops other slots first.
2. `narrator:time_and_emotions` — state lines only; **when `FUROSHIKI_DIRECTIVE_FIRST=1`, promoted to Band 1** after `live_time_anchor` when the narrator block is non-empty; otherwise remains here after the tone block.
3. `narrator:tasks` — compressed.
4. `session_thread_grounding` — verbatim user thread + current turn.
5. `context_augmentation:*` — time-bounded factual augmentations.
6. `user_need_directives`.
7. `conversation_compass` — was landing *after* channel contract in V1; highest-leverage single move.
8. `thread_continuity_priority` — last rule before static tail.

Overflow policy: composite truncates first; then oldest session-thread turns drop. If composite + thread grounding alone exceed cap, log a warning — resolver is over-producing upstream.

### Band 4 — Static Tail Anchor (unchanged, target ~1,100 tok)

Tool-use rules with maximum recency. **No turn-variable content enters Band 4.** Position is load-bearing.

## 5. Separator discipline

V1 uses `---` for every break, flattening hierarchy. V2 uses two delimiter levels:

- **Band boundary:** `\n===\n` (three boundaries total).
- **Section within a band:** `\n---\n` (existing).

Models treat delimiter hierarchy as structural signal. Costs ~12 tokens; makes the two-anchor shape legible to the model.

## 6. Behavioral-path audit

Every Band 3 piece has write → read → context path:

| Piece | Write site | Read cue |
|---|---|---|
| composite_voice_directive | resolver in `voice_context` | directs tone explicitly |
| operator_corrections | `db.get_recent_operator_corrections` → `voice_context.build_operator_corrections_block` | authoritative wrong→right corrections from the operator |
| narrator:time_and_emotions | soul engine tick | referenced by composite directive |
| session_thread_grounding | listener | `thread_continuity_priority` instructs use |
| user_need_directives | `user_needs_model` | self-labeled directives |
| conversation_compass | `context_zones.build_conversation_compass` | topic steering |
| context_augmentation:* | plugin hooks | time-bounded factual use |

Band 2 structured blocks gain a framing header in V2; without it they fail the read-side test.

## 7. Contract for the composite voice directive

V2 depends on the resolver from `EMOTIONAL-DIRECTIVE-OVERHAUL.md`:

- **Output:** one text block, ≤ 300 tokens, no internal "HARD RULE"/"NEVER" preambles.
- **Inputs:** current `behavioral_cues` inputs + mind-state suppression rules + `mood_score` + conflict resolution across stacked emotions.
- **Determinism:** pure Python. No LLM call.
- **Fallback:** if resolver errors, V2 falls back to V1's `behavioral_cues` block placed in Band 3 (not Band 1), with a logged warning.

Failure mode to watch: if resolver can't meet 300 tok budget, Band 3 exceeds cap and channel contract loses recency grip.

## 8. Token budget

| Band | V1 observed | V2 target | V2 hard cap |
|---|---|---|---|
| 1 Top | ~1,100 | ~1,200 | 1,400 |
| 2 Mid | ~800 | ~1,200 | 1,500 |
| 3 Turn-live | *(distributed across middle)* | ~500 | 600 |
| 4 Tail | ~1,100 | ~1,100 | 1,200 |
| **System total** | **~6,100** | **~4,000** | **4,700** |

Net ~2k-token reduction because the composite resolver collapses three overlapping tone blocks into one.

## 9. Migration & rollout

1. **Land resolver first.** V2 cannot enable without the composite voice directive resolver. Sequence: overhaul Phases 1 + 4 → V2 flag work.
2. **Implement behind flag.** `CONTEXT_ASSEMBLY_V2` in `context_zones.py`. V1 path untouched and reachable.
3. **Trace parity run.** For 24h with flag off, log what V2 *would have* produced (dry-run assembler) alongside V1. Diff token counts, band sizes, piece placement. Catches resolver overflow before it reaches the model.
4. **Flag on.** One-week soak with both rubrics watched:
   - Warmth rubric (primary signal from overhaul work) — must improve or V2 isn't earning its keep.
   - Tool-call correctness — must not regress; channel contract position is the guard.
5. **Default-on.** Keep V1 path one release cycle; delete after.

Rollback: flip flag off. No data migration, no schema to unwind.

## 10. Risks

- **Resolver overshoots 300 tok.** Band 3 bloats, channel contract loses grip, tool-use regresses. Mitigation: hard cap + dry-run parity.
- **Mid band becomes a dumping ground.** Future pieces default to Band 2 because it has no identity. Mitigation: 1,500-tok hard cap enforced in code, not guideline.
- **Band 3 ordering becomes load-bearing in subtle ways.** Reordering within Band 3 is a V2.x spec change, not a free edit.
- **Separator change confuses trace parsers.** Verify downstream trace consumers parse the new `===` boundary.

## 11. Documentation touch list

- `DYNAMIC-CONTEXT-LAYERS.md` — replace V1 zone 1/2/3 description with four-band model; document separator convention.
- `EMOTIONAL-DIRECTIVE-OVERHAUL.md` — note resolver output lands in Band 3, bounded ≤ 300 tok; add fallback contract.
- `TOKEN-LIMITS.md` — per-band budgets and caps.
- `INTEGRATION.md` — confirm harness-agnostic: bands assembled before handoff to any channel adapter.
- `PLUGIN-SPEC.md` — plugin pre_conversation content lands in Band 1; time-bounded augmentations in Band 3.

## 12. Open question (non-blocking)

Does `context_augmentation:design_docs` (observed in trace 3) ever carry content that should persist beyond the current turn? If yes, it needs a dual-write story — Band 3 for the live turn, Band 2 feeder for subsequent turns. If no (true ephemeral), current Band 3 placement is complete. Defaulting to "no" until a use case surfaces.

**Expiry timing note (re-examine before V2 flag-on):** A production trace showed `context_augmentations: expired 1 unpinned row(s)` immediately before a conversation turn where the user asked about a proactive message. If ephemeral augmentations can carry "here's what the last proactive was about" content, their expiry window must exceed the typical user-response latency (often hours). If they expire first, the grounding is gone exactly when most needed. Resolution: if augmentation content is proactive-response grounding, it should either (a) have a longer TTL tied to `last_proactive_outreach.surfaced_at` + some hours, or (b) be sourced from heartbeat directly (see §13 below).

---

## 13. Gap: no `last_proactive_outreach` piece in the listener

**Problem.** The heartbeat stashes `last_proactive_outreach` (kind, summary, detail, telegram_preview) after every proactive send. The chat listener never injects any of this. When the user responds to a proactive, the model must reconstruct from history alone what that outreach was actually about — and history contains the composed telegram text, which may have had wrong attribution (see production bug: `design_idea` framed as "your idea").

**Proposed fix.** Add a `proactive_context` piece to Band 3, bounded ≤ 80 tok, that fires only when:
1. `last_proactive_outreach.surfaced_at` is within the last 6 hours, **and**
2. the current turn looks like a response to the proactive (micro-brain signal, or recency heuristic).

Content shape:
```
[LAST PROACTIVE — context for this turn]
kind: design_idea
source: your own reflection (not something the user said)
summary: Add a capability vs. proposal flag on ideas you generate…
sent: 6h ago
```

This gives the model the ground-truth source without relying on the possibly-wrong composed text.

**Behavioral-path entry** (for §6 audit table):

| Piece | Write site | Read cue |
|---|---|---|
| `proactive_context` | `voice_dispatcher.record_last_proactive_outreach_context` → heartbeat | Kind and source field explicitly frame attribution |

---

## 14. Gap: `session_thread_grounding` propagates bad proactive framing

When a proactive DM misattributes an idea to the user and the user then repeats that framing ("that idea I had"), `session_thread_grounding` gives the user's message "highest priority — do not re-quiz" authority. The model reads the user's parroted framing as an authoritative user-stated fact.

**Mitigation options (pick one before V2 flag-on):**
1. **Preferred:** inject `proactive_context` (§13) earlier in Band 3 than `session_thread_grounding` so the source framing supersedes the parroted text.
2. **Alternative:** annotate history turns written by `source='proactive'` so the grounding block can exclude the immediately following user turn from "authoritative user-stated fact" status when a proactive is recent.
3. **Fallback:** no code change — rely on the P0 outreach fix (attribution note in `build_prompt`) preventing the bad text from reaching the user in the first place.

Option 3 is the current state (P0 is now shipped). Options 1 and 2 are defense-in-depth.

---

## 15. Gap: Band 2 framing headers underspecified

§4 states "Structured blocks gain a one-line framing header so they have a read-side in the prompt." The spec does not define what those headers are.

**Required before implementation:**

| Block label | Proposed framing header |
|---|---|
| `structured:user_anticipations` | `ANTICIPATED USER NEEDS — what Furoshiki expects the user may want or ask; use to anticipate, not to script.` |
| `structured:emotional_weights` | `EMOTIONAL SIGNAL WEIGHTS — how strongly recent events should color tone; no quality_scoring entries should appear here.` |
| `structured:conversation_mode` | `CONVERSATION MODE — current topic classification guiding which context is most relevant.` |

Each header is one line, ≤ 20 tok. The header is what makes the block "behavioral-path complete" per §6. Without it, the model has no read-cue to act on the block.

**`structural:emotional_weights` quality-scoring leak:** CLAUDE.md flags `source_session = 'quality_scoring'` entries as red flags that must not reach the prompt. V2 moves this block into Band 2 but does not fix the upstream leak. That leak is tracked separately; V2 should add an assertion: before inserting this block, strip any entry where `source_session = 'quality_scoring'` and log a warning.

---

## 16. Gap: behavioral-path audit table incomplete (§6 update)

Corrected and completed table:

| Piece | Write site | Read cue |
|---|---|---|
| `composite_voice_directive` | resolver in `voice_context` | directs tone explicitly |
| `narrator:time_and_emotions` | soul engine tick | referenced by composite directive |
| `narrator:tasks` | `refresh_session_context` → task list query | explicit task-state lines; model acts on pending items |
| `session_thread_grounding` | listener (per-turn) | `thread_continuity_priority` instructs use |
| `context_augmentation:*` | plugin `post_inbound_signals` hooks (time-bounded) | time-bounded factual injection; ephemeral by design |
| `user_need_directives` | `user_needs_model.build_user_need_directives` | self-labeled directives |
| `conversation_compass` | `context_zones.build_conversation_compass` | topic steering |
| `proactive_context` (proposed, §13) | `voice_dispatcher.record_last_proactive_outreach_context` → heartbeat | Kind and source field frame attribution |

---

## 17. Gap: dry-run parity runner format undefined

§9 step 3 requires logging "what V2 would have produced alongside V1." Without a defined format this step will be skipped or done inconsistently.

**Proposed format.** Emit one JSON object per turn to `logs/context-trace-v2-dryrun.jsonl` with:

```json
{
  "ts": "ISO8601",
  "user_message_sha256": "...",
  "v1_band_sizes": {"band1_tok": 1100, "band2_tok": 800, "band3_tok": 1400, "band4_tok": 1100},
  "v2_band_sizes": {"band1_tok": 1180, "band2_tok": 950, "band3_tok": 490, "band4_tok": 1100},
  "v1_total_tok": 4400,
  "v2_total_tok": 3720,
  "v2_overflow": false,
  "v2_fallback_used": false,
  "pieces_v2": ["identity_shell", "live_time_anchor", ...]
}
```

The `furoshiki trace` command should be able to diff two runs by sha256. The dry-run writer lives in `context_zones.py` alongside the assembler and is gated by `CONTEXT_ASSEMBLY_V2_DRYRUN=1`.

---

## 18. Gap: V2 fallback path underspecified

§7 describes the fallback when the composite resolver errors. Two flags interact (`CONTEXT_ASSEMBLY_V2`, `FUROSHIKI_COMPOSITE_DIRECTIVES`) and the exact behavior when V2 is on but composite is off is unspecified.

**Explicit contract:**

| V2 flag | Composite flag | Band 3 behavior |
|---|---|---|
| off | any | V1 assembly unchanged (behavioral_cues in Band 1 position) |
| on | on | `composite_voice_directive` / `behavioral_directives` in **Band 1** when `FUROSHIKI_DIRECTIVE_FIRST=1` (default), else Band 3 slot 1 |
| on | off | V1 `behavioral_cues` placed in Band 3 slot 1 (not Band 1) when directive-first is off; log `WARNING context_assembly_v2: composite_directives flag off — using legacy behavioral_cues in Band 3` |
| on | on, resolver errors | Same as (V2 on, composite off) with `ERROR` log |

Band 3 ordering is load-bearing (§10). The fallback must preserve the order: behavioral_cues first, then remaining pieces in spec order. Placing it in Band 1 would undo the structural benefit of V2.

---

## 19. Gap: separator change and downstream trace consumers

`===` band boundaries will break any code that splits system prompts or traces by `---` alone.

**Files to audit before V2 flag-on:**

- `scripts/dashboard_data.py` — parses system prompt content from context traces (multiple `re.MULTILINE` splits). The `===` boundary must be recognized as a superset delimiter; content between `===` markers should still be split by `---`.
- `scripts/dashboard_routes.py` — passes trace data to the frontend; verify it doesn't assume a flat structure.
- `furoshiki trace` CLI — must display band boundaries visually. Recommended: display `===` as a thick rule in terminal output.
- `tests/` — any snapshot or golden-file tests that include system prompt text.

---

## 20. Outreach pipeline gap (independent of V2 scope)

V2 only covers `telegram_listener.build_system_prompt`. The outreach composer (`voice_dispatcher.build_prompt`) has the same structural problem: hardcoded kind-specific notes, no template system, limited sanitization.

### 20.1 P0 fix (shipped)

`design_idea_note` added to `build_prompt()` for `_SELF_ORIGINATED_OUTREACH_KINDS`. Extended `sanitize_proactive_outreach_attribution()` to catch "I saw your idea", "your idea about", "you thought of", "you came up with" patterns. `outreach_truth_tier()` now returns `'self_originated'` for these kinds.

### 20.2 Template-based outreach kind notes and domain-adaptive context shape — V2.1

Full spec: **[`CONTEXT-ASSEMBLY-V2.1.md`](./CONTEXT-ASSEMBLY-V2.1.md)**

Summary:
- **Phase 1** — Extract hardcoded kind notes from `build_prompt()` to `config/outreach_kind_templates.json`. Config-driven, no behavioral change on first deploy, dashboard-editable. No flag needed.
- **Phase 2** — Context role profiles (`config/context_role_profiles.json`) derived from `conversation_mode` + `active_domain`. Adjusts V2 band budget caps and piece inclusion per turn. Gated by `CONTEXT_ASSEMBLY_V2_1` flag; requires V2 and a domain-detection accuracy baseline (≥ 85%).

These are non-blocking for V2. V2 ships and stabilizes first; V2.1 phases follow in the sequence documented in that spec.
## 20. Modular assembly (`scripts/prompt_builder.py`)

V2 band construction lives in `prompt_builder.py`: `build_band1` … `build_band4_trailing`, plus `assemble_v2_system_prompt`, which `telegram_listener.build_system_prompt` calls when Context Assembly V2 is on.

- Each band builder has a single documented purpose (see docstrings in that module).
- Every context piece dict includes integer `band` (1–4) for dashboard traces and `furoshiki trace --full` grouping.
- **`FUROSHIKI_DIRECTIVE_FIRST`** (default on): behavioral directives and `narrator:time_and_emotions` move into Band 1 immediately after the identity shell so primacy bias favors tone before factual dossier. When set to `0`, behavioral directives stay in Band 3 slot 1 (legacy V2 ordering).
