Metadata-Version: 2.4
Name: context-backpressure
Version: 0.1.0
Summary: A context pressure governor for coding-agent loops: detect rising context pressure in recorded agent traces and decide when to page in, pin, evict, compact, verify, reset, split, or stop.
License-Expression: MIT
Project-URL: Homepage, https://github.com/zihangzzz/context-backpressure
Project-URL: Repository, https://github.com/zihangzzz/context-backpressure
Project-URL: Issues, https://github.com/zihangzzz/context-backpressure/issues
Project-URL: Documentation, https://github.com/zihangzzz/context-backpressure/tree/main/core/docs
Keywords: agents,context,backpressure,llm,coding-agents
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# Context Backpressure

**v0.1.0 — Public Alpha / Research Preview.** The core is deterministic and
usable for local evaluation, but its interfaces and calibration may change as
evidence grows. It is not a production-stability or population-accuracy claim.

The PyPI package contains the portable, vendor-neutral core CLI and library.
The complete desktop Sidecar bundle, session launchers, and harness adapters
live in the [GitHub repository](https://github.com/zihangzzz/context-backpressure).

**Coding agents don't just run out of context — they drown in it. This is a governor that detects rising context pressure in agent loops and decides when to push back.**

## What this is

Long-running coding agents rarely fail because they *can't* get context. They fail because their loops keep *accepting* context after its marginal value has gone negative: broad "just in case" file reads, stale assumptions, verbose tool outputs, and failed-attempt residue all stay resident, and the loop pays for them in cost, distraction, and drift.

Context Backpressure is a small, verifiable **runtime-control layer** for coding-agent loops. Given a recorded JSONL trace of an agent run, it:

1. maintains a **context ledger** — every context item gets a *lease*; references renew it, silence expires it, pins exempt invariants;
2. computes a multi-signal **context pressure index** — staleness, absolute dead rent, redundancy, overfetch, failure repetition, verification stagnation, plan drift, and token utilization;
3. emits **governor decisions** from an ordered, auditable rule table: allow, demand page-in justification, evict, compact, pin, force verification, recommend reset / split / stop-and-ask-human;
4. records a separate **attention state** that budgets human interruption with persistence and hysteresis, without hiding or changing the raw pressure or decision.

It is model-agnostic, tool-agnostic, CLI-first, offline-first, and dependency-free (stdlib + pytest).

## What this is not

| Not this | Because |
|---|---|
| Prompt/context engineering | That is assembly-time and static; this operates on the *loop trajectory* at run time. A static context file is a baseline this project measures against. |
| An agent harness | A harness owns the loop. This owns nothing: it consumes a trace and emits a decision. Harness : governor :: kernel : cgroup controller. |
| Loop engineering | Plan-act-verify structure is orthogonal. This asks whether the loop's *memory* is still fit to decide anything. |
| Context compression | Compression is one actuator. This is the controller deciding *when* compaction (or eviction, or refusal-to-ingest) is warranted — and when it won't be enough. |
| A statusline / context visualization | Gauges leave the decision to a human. This computes a machine-actionable `PolicyDecision` with a rule-traceable justification. |
| A RAG system | Nothing here retrieves. It governs what retrieval already brought in. |

## Why token budget is not context health

Token count is a *capacity* signal. A loop at 40% of budget can already be in trouble — re-reading the same file, retrying a failed approach, hauling dead context through every step. A loop at 85% with pinned invariants and improving tests can be perfectly healthy. That is why token utilization keeps its 0.15 raw weight and is interpreted beside seven other health signals; utilization never decides an action by itself.

## Architecture

```
 JSONL trace ──▶ Trace Reader ──▶ per step: ┌───────────────────────────┐
 (+ .meta.json)                             │ Context Ledger            │
                                            │  items · leases · pins    │
                                            └────────────┬──────────────┘
                                                         ▼
                                            ┌───────────────────────────┐
                                            │ Pressure Engine           │
                                            │  8 signals → index → zone │
                                            └────────────┬──────────────┘
                                                         ▼
                                            ┌───────────────────────────┐
                                            │ Governor (rule table)     │
                                            │  R1..R10 → PolicyDecision │
                                            └────────────┬──────────────┘
                                                         ▼
                                            ┌───────────────────────────┐
                                            │ Attention replay          │
                                            │ dwell · hysteresis ·      │
                                            │ current episode           │
                                            └────────────┬──────────────┘
                                                         ▼
                              scorer (12 metrics) · Markdown report
                              session export (cbp-session/v1) ─▶ Vitals Monitor
                                        static replay (cbp monitor)
                                        live loopback sidecar (cbp watch)
```

Everything is a deterministic fold over replayed state; the ledger and attention stream are the two explicit state machines. Nothing calls a model, and nothing leaves the machine: the core is fully offline, and the live sidecar is a stdlib loopback server (127.0.0.1) owned by your own CLI process for the length of one session.

## Install the core CLI

```bash
python -m pip install context-backpressure
cbp --help
```

The core has no runtime dependencies. The repository checkout is required for
the bundled fixtures, desktop companion, and live-session launchers.

## Source checkout and fixtures

Fresh checkout (requires Python ≥ 3.10; `python3` is the only assumed command):

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python -m pytest              # 569 tests, loopback only, no display needed

cbp score     fixtures/traces/stale_accumulation.jsonl
cbp recommend fixtures/traces/stale_accumulation.jsonl
cbp explain   fixtures/traces/stale_accumulation.jsonl --step 13
cbp compare   fixtures/traces/healthy_short.jsonl fixtures/traces/stale_accumulation.jsonl fixtures/traces/plan_drift.jsonl
cbp compare   fixtures/traces/healthy_short.jsonl fixtures/traces/stale_accumulation.jsonl --markdown --out reports/comparison.md
cbp report    fixtures/traces/stale_accumulation.jsonl --out reports/stale_accumulation.md
cbp monitor   fixtures/traces/majority_stale.jsonl   # → majority_stale.monitor.html
```

Full live loop, two terminals (an agent stand-in feeds a recorded trace in real time):

```bash
cbp pet live.jsonl &                                    # the desktop pet (or: cbp watch)
cbp feed fixtures/traces/majority_stale.jsonl live.jsonl  # the "agent"
# watch recorded attention rise as evidence persists, click to inspect or
# respond to a current reset episode; `cbp watch live.jsonl --open` serves the same
# state as a browser page — then Ctrl-C → live.report.md + live.monitor.html
```

`cbp` is the console script installed by pip. If it is not on your PATH (or you skipped the editable install), the module form is equivalent:

```bash
python3 -m context_backpressure.cli score fixtures/traces/stale_accumulation.jsonl
```

`cbp explain TRACE [--step N]` defaults to the final recorded step. It shows
each recorded signal, measured flag, configured weight, literal weighted
numerator term, and normalized contribution (the eight contributions add
directly to the displayed pressure),
the fixed numerator/denominator, ledger absolutes, actual pre-step context
telemetry when available, attention state, and the governor's verbatim
decision justification. Unknowns remain `?`; this presentation command does
not compute a signal or run a second policy engine.

`cbp compare` accepts two or more traces. Its default is the compact terminal
table; `--markdown` renders the same metrics as a portable N-way Markdown table,
and `--out` writes the selected format instead of printing it. When a compared
trace has a sibling `<trace>.acks.jsonl`, both renderers add per-rule alarm
response counts: acknowledged, dismissed, and unaddressed. "Judged precision"
is acknowledged / (acknowledged + dismissed), with unaddressed alarms shown
separately; a missing log is reported as missing rather than silently treated
as operator silence. These clicks remain exploratory operator judgements, not
proof that a follow-up action occurred or that a threshold is calibrated.

## Example output

`cbp recommend` on the stale-accumulation fixture (an agent front-loads six file reads, uses two, and drags four dead files through the run):

```
step  zone    pressure  action
   0  green       0.00  allow_continue
   1  green       0.13  allow_page_in
   ...
   3  yellow      0.16  require_page_in_justification  — rule R8: zone yellow, page-in
                        requested, overfetch_ratio 0.13 >= 0.13  [targets: f_middleware]
   ...
  12  yellow      0.36  allow_continue
  13  orange      0.39  evict_stale  — rule R4: zone orange and stale_fraction 0.44 >= 0.3;
                        3 expired unpinned item(s)  [targets: f_config, f_middleware, f_session]
                        [~2100 tokens freed]
```

The governor names exactly the three files whose leases lapsed unreferenced. On the healthy control fixture it stays silent for the whole run — silence on healthy loops is a feature, and there is a test for it.

## The Vitals Monitor: replay, live sidecar, and desktop pet

The metaphor is clinical telemetry. A **hero readout** always shows the exact raw pressure and its analytical zone; current chrome follows the separately recorded attention zone. The pressure trace scrolls over raw zone bands; a **verification pulse lane** draws a heartbeat blip for every test run (a verification blackout literally flatlines, in amber dashes); repeated failures tick like an arrhythmia; and current governor episodes surface by action urgency — notices, maintenance advisories, and human gates — each carrying its targets, estimated tokens freed, and the rule-traceable justification verbatim.

It runs in two modes:

- **`cbp monitor <trace>`** — a self-contained HTML replay: one file, inline everything, openable from disk, makes no requests of any kind (tested).
- **`cbp watch <trace>`** — the **live sidecar**. Point it at a JSONL trace that is still being written (start it before the agent: it shows a waiting state) and it serves the same page in live mode over 127.0.0.1: a LIVE chip follows the newest step, scrubbing back pauses following, and active maintenance/human episodes grow **acknowledge / dismiss** buttons. Operator responses append to `<trace>.acks.jsonl`; ending the session (Ctrl-C or `kill`) writes a post-session report with an **Operator Response** table — acknowledged, dismissed, or *unaddressed*, per alarm — plus a static monitor carrying separate response and resolution history. That closes the loop: live trace → raw measurement + decision → attention state → operator action or silence → logged response → post-session report.
- **`cbp pet <trace>`** — the same live state as a **desktop pet**: a small frameless always-on-top orb (stdlib Tk, no server, no sockets — it watches the trace file in-process). The orb is a *black hole of context*: every resident context item orbits the core as a particle — fresh ones bright, stale ones ash-gray (dead mass still orbits, still costs), pinned invariants gold and steady, duplicates flickering. Its exact number remains raw pressure; its colour, orbit, spin, breathing, collapse, and current card follow recorded attention. A critical episode relaxes when its evidence resolves, independently of whether anybody clicked it. Click the orb to inspect the raw/attention state or respond with the same acknowledgement log used by reports. It dozes when its agent goes idle.

**Which agent is which?** One pet (or sidecar) binds to exactly one trace — never a multiplexer. Three pairing mechanisms: a deterministic **two-letter sigil** derived from the trace path, worn by the pet and printed by `cbp watch` in the agent's terminal, so you match by glancing; a **live ticker** under the orb showing the newest step's intent, so the pet visibly mirrors what its agent just did; and **spatial pairing as your gesture** — drag the pet next to its agent's window, and the position persists per-trace across launches. Run five agents, get five orbs, each wearing its sigil.

`cbp feed` is the demo's agent stand-in: a *replay pump* that drips an already-recorded trace into a file in real time. It invents nothing — it is a pacing device, not a simulator.

## Privacy

The core makes no hosted or outbound request, but its inputs and generated
artifacts are not automatically sanitized or encrypted. Traces,
acknowledgement logs, Markdown reports, and self-contained HTML monitors may
contain task text, session identifiers, local paths, commands, tool arguments,
and tool output inherited from the source log. Review and redact an artifact
before sharing it. The standalone CLI writes to operator-selected paths and
may place a report beside its trace or a monitor in the current directory;
only the cross-platform bundle launcher confines its runtime data to `.cbp`.

Three product laws keep it honest:

1. **The monitor is a renderer, never a second policy engine.** Every pixel traces to a recorded `PressureState`, `PolicyDecision`, `AttentionState`, ledger digest, or acknowledgement event exported by `cbp export` (`cbp-session/v1` JSON). It computes no threshold, dwell, signal, or rule. The sidecar computes nothing new either — it re-replays the growing trace through the exact pipeline `cbp report` uses.
2. **Silence is the default state.** On the healthy control fixture the monitor shows a flat green trace and the line *"Silent run — the governor allowed every step."* — and there is a test for it.
3. **Nothing hosted, nothing remote.** The static file makes zero requests (tested: no `fetch` in the artifact). Live mode talks only to the loopback sidecar owned by your own `cbp watch` process — three endpoints (page, session, ack), alive for one session, then gone.

Unknowns still look unknown everywhere: sparse verification evidence renders as `stagnation ?`, never as a measured value; unmodeled relief renders as `n/a`, never as a guess.

Try `cbp monitor` on each fixture — the eight-fixture corpus is the demo reel.

## Metrics

Per trace, `cbp score` computes: task success, steps to green, token budget used, mean active context tokens, file overfetch ratio, stale context count, stale context half-life, repeated failure count, plan drift count, verification progress slope, and mean/peak context pressure index. Definitions live in [docs/eval-methodology.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/eval-methodology.md).

## Fixtures as failure taxonomy

Eight hand-authored traces, each making one behavior obvious (see [docs/failure-taxonomy.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/failure-taxonomy.md)). Every governor rule has trace-level fixture evidence:

| Fixture | Failure mode | Expected governor behavior |
|---|---|---|
| `healthy_short` | none (control) | silence: allow/page-in only |
| `stale_accumulation` | dead context riding along | `evict_stale` naming the dead files |
| `repeated_failure` | same failure four times | `stop_recommended` (human gate) |
| `overfetch_spiral` | "might be relevant" intake habit | `require_page_in_justification` |
| `plan_drift` | goal quietly mutating | `split_task_recommended` |
| `tool_output_flood` | verbose logs kept resident forever | `compact_tool_outputs` on the old logs |
| `majority_stale` | working set goes majority-dead | `reset_recommended`, carrying pins forward |
| `verification_blackout` | 13 steps with zero verification signals | `force_verification` under pressure |

## Limitations (honest version)

- **The governor is advisory, and systematic policy-efficacy evidence is still thin.** The generic adapter (`cbp adapt`, [docs/adapters.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/adapters.md)) has supported several healthy shadow pilots and a captured long trace that exposed formula-shape defects. Those runs validate observability and falsification, not the counterfactual claim that following an alarm improves an outcome. A labeled failing session with action/no-action comparison is still missing.
- Fixture traces are hand-authored to make failure modes legible. They are unit tests for the formula, not evidence about production agents.
- Exploratory operator acknowledgements exist, but they do not establish calibrated alarm precision or prove that a follow-up action occurred.
- Signals use cheap proxies (exact-hash redundancy, lexical anchor-containment drift, string-matched failure signatures). Exact hashes miss near-duplicates; containment can miss a semantic reversal that reuses anchor words and can overstate a faithful paraphrase that introduces new vocabulary; sticky verification exposes `evidence_age_steps` but cannot prove that old evidence is semantically fresh.
- The raw 0.15/0.38/0.45 zones and R8 cutoff 0.13 are fixture- and pilot-calibrated. The human-facing attention bands are separately replay-calibrated at yellow 0.30/0.26, orange 0.38/0.34, and red 0.45/0.41 with four-frame ordinary rise dwell. Per-harness deployment calibration remains an evidence task; neither layer is a population-level statistical claim.
- **Reference inference is a measured heuristic.** The report metric `file_overfetch_ratio` was 0.68 on the 427-step build trace; it is a whole-run fetched-token metric, not formula v1's fixed-budget resident-residue signal. Path mentions credit only the newest resident occurrence and can miss unrecorded use, so both metrics can still read high when the trace cannot see a real reference.
- A mock-agent simulator with a fault model and closed-loop policy comparison remain deliberately future work — see [docs/roadmap.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/roadmap.md).

## Docs

- [docs/HANDOFF.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/HANDOFF.md) — **start here if you're new**: thesis, taste constraints, invariants, product direction, review checklist
- [docs/thesis.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/thesis.md) — the one-page argument
- [docs/architecture.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/architecture.md) — components, schemas, data flow
- [docs/eval-methodology.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/eval-methodology.md) — metric definitions, fixture assertions, honesty measures
- [docs/adapters.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/adapters.md) — the generic `toolcall/v1` format, mapping rules, and how to write a shim
- [docs/shadow-protocol.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/shadow-protocol.md) — the pre-registered shadow-mode pilot: what "going sideways" and "a useful alarm" mean, decided before the data
- [examples/shadow/](https://github.com/zihangzzz/context-backpressure/tree/main/core/examples/shadow) — a fully worked synthetic example of the labeling and scoring workflow, kept current by test
- [docs/failure-taxonomy.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/failure-taxonomy.md) — the five failure modes and their signal signatures
- [docs/roadmap.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/roadmap.md) — four-week extension plan
- [docs/interview-pack.md](https://github.com/zihangzzz/context-backpressure/blob/main/core/docs/interview-pack.md) — narrative, design decisions, anticipated questions
