Metadata-Version: 2.5
Name: jev-gc
Version: 0.1.0
Summary: Real-time OpenTelemetry-based context garbage collection for LLM agents, powered by Jev (TypeSafe AI's System One model).
Project-URL: Homepage, https://adigulalkari.github.io/Jev_GC/
Project-URL: Documentation, https://adigulalkari.github.io/Jev_GC/
Project-URL: Repository, https://github.com/adigulalkari/Jev_GC
Project-URL: Issues, https://github.com/adigulalkari/Jev_GC/issues
Author-email: Adi Gulalkari <adigulalkari825@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,context-management,jev,langgraph,llm,opentelemetry,token-optimization,typesafe
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: cachetools<6,>=5.3
Requires-Dist: httpx<1,>=0.27
Requires-Dist: opentelemetry-api<2,>=1.24
Requires-Dist: opentelemetry-sdk<2,>=1.24
Requires-Dist: pydantic-settings<3,>=2.2
Requires-Dist: pydantic<3,>=2.6
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: tenacity<9,>=8.2
Provides-Extra: dev
Requires-Dist: freezegun>=1.4; extra == 'dev'
Requires-Dist: mypy>=1.9; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: examples
Requires-Dist: httpx<1,>=0.27; extra == 'examples'
Requires-Dist: mistralai>=1.0; extra == 'examples'
Requires-Dist: strands-agents>=0.1; extra == 'examples'
Provides-Extra: langgraph
Requires-Dist: langgraph>=0.2; extra == 'langgraph'
Provides-Extra: strands
Requires-Dist: strands-agents>=0.1; extra == 'strands'
Description-Content-Type: text/markdown

# jev-gc

Real-time, OpenTelemetry-driven context garbage collection for LLM agents,
powered by [Jev](https://typesafe.ai) (TypeSafe AI's "System One" model).

**[Documentation and measured overhead →](https://adigulalkari.github.io/Jev_GC/)**

## The problem

Long agent sessions accumulate tool results, retrieved chunks, and failed
calls faster than any prompt window can hold. All of it gets re-sent to
the LLM on every turn whether or not it's still relevant — wasting tokens
and burying what actually matters under stale noise ("context rot").

## What jev-gc does

Hooks into the OTel spans your agent already emits and decides, span by
span, in real time: keep verbatim, compress to a pointer, or archive.
Nothing is ever hard-deleted. Cheap deterministic rules (pins, errors,
recency, staleness) resolve most spans for free — only the genuinely
ambiguous ones go to Jev for a relevance/treatment judgment.

```
Agent tool call → OTel span emitted → jev-gc classifies → next prompt
                                            │
                           deterministic rules resolve most spans free;
                           only the ambiguous ones reach Jev
```

## Why a keyword rule isn't enough

Real output from [`scripts/relevance_demo.py`](./scripts/relevance_demo.py)
against the live Jev API. Task: *"Reconcile invoice #4521 totals across
the ERP system and warehouse shipment records."* Two tool results, same
tool, same shape — one caused the mismatch, one is unrelated noise:

| span | keyword-overlap score | Jev relevance score |
|---|---:|---:|
| shipment delay for *this* order | 0.11 | **0.75** |
| shipment delay for an *unrelated* order | 0.11 | **0.45** |
| line-item breakdown from 3 turns ago | 0.33 | **0.97** |

A keyword/BM25 rule gives the relevant and irrelevant spans the **same
score** — it has no way to know one belongs to invoice #4521 and the
other doesn't; that's a fact about meaning, not vocabulary. Jev separates
them by 0.30. Reproduce it: `export JEV_API_KEY=... && python scripts/relevance_demo.py`

## Real run

[`examples/codebase_triage_agent/`](./examples/codebase_triage_agent/): a
bug-triage agent that investigates real bug reports against this actual
repo's own source, tests, and git history — real `git grep`, real file
reads, real `pytest` runs, no mocked data — through jev-gc end to end
against the live Jev API. See that example's README for the real
`tokens_saved_estimate` numbers from a full run.

## Design

1. **Code calculates, Jev judges.** Thresholds/age/status/pins are plain
   code. Jev only sees what's left.
2. **Fail open.** Jev down, slow, or low-confidence → keep at current tier.
3. **Never hard-delete.** Evicted just means out of the *next* prompt.
4. **Errors are never dropped, only compressed.** Only payload size is
   negotiable via Jev — presence never is.

`JevClient` is a `Protocol`, not a hard dependency threaded through the
codebase — swap in another scorer behind the same interface if you want.
Full rationale in [`SPEC.md`](./SPEC.md).

## Install

```bash
pip install -e ".[dev,langgraph,strands]"
```

## Quickstart

```python
from opentelemetry.sdk.trace import TracerProvider
from jevgc import JevGC

provider = TracerProvider()
gc = JevGC.from_config("jevgc.yaml")            # reads JEV_API_KEY from env
gc.attach_to_tracer_provider(provider)

# ... agent runs, spans stream in automatically ...

prompt_context = gc.build_context(
    task="Reconcile invoice #4521 totals across ERP and warehouse",
    budget_tokens=8000,
)
```

No OTel yet? Use the manual escape hatch: `await gc.observe(span_record)`.
See [`docs/quickstart.md`](./docs/quickstart.md).

## Getting evicted context back

Eviction would be a one-way door if the only way to ask for a span were a
`span_id` the agent can no longer see. So every span leaving HOT is archived
as an immutable snapshot, and a keyword-only index of what's evicted stays
cheap enough to show the agent every turn:

```python
gc.cold_index()                      # keywords only, never content
gc.search_cold("rotterdam manifest") # -> [ColdIndexEntry(span_id=..., tier=COLD)]
gc.rehydrate(span_id)                # full content back in HOT
```

`rehydrate` resolves to the snapshot taken when the span was observed, not to
a live re-read of the original source — what comes back is what was actually
evicted, so an audit of that decision reads the same evidence the decision saw.

Retaining that content costs memory, so the archive runs under a budget
(`archive.max_content_bytes`, default 8 MiB). Past it, the least-recently-used
snapshots release their *content* and keep their *keywords*: the span stays in
the cold index, still answers a search, and still counts in a regret pass — it
just can't be restored verbatim anymore, and `rehydrate` degrades to a tier
promotion with a logged warning. Keywords cost ~100 bytes against content that
runs to kilobytes, so discoverability outlives restorability by a wide margin.
`gc.archive_stats()` reports `retained_bytes` and `released_count`; a
`released_count` above zero means you've hit the ceiling.

## Measuring false eviction

The failure mode that matters isn't a bad summary, it's a span that looked
irrelevant at turn 3 and mattered at turn 11 — silently, with no error. Every
tier transition is logged, and a shadow replay diffs a no-eviction baseline run
against the GC'd run:

```python
findings = gc.analyze_regret(
    baseline_output=full_context_run_answer,
    evicted_output=gc_run_answer,
)
```

Each `RegretFinding` names an evicted span whose distinctive terms surfaced
only in the baseline answer. It's keyword-overlap evidence, not proof of
causation — a ranked list of evictions worth inspecting and a dial for tuning
`relevance_keep_threshold`, not a regression gate.

Note what this shadow replay does *not* compare: prompt-cache usage between
the baseline and GC'd run. Under a prefix-based cache, evicting or rewriting
an early span changes every later byte's position in the prompt, so the
unchanged tail can miss the cache too — a shorter, GC'd prompt is not
automatically a cheaper one if it keeps breaking cache reuse that a stable,
never-evicted prompt would have kept hitting. `analyze_regret` and the
benchmarks below report token counts, not cached-vs-uncached input tokens, so
neither currently answers "does this save money," only "does this save
tokens" and "did an eviction visibly cost an answer."

## Docs

- **[adigulalkari.github.io/Jev_GC](https://adigulalkari.github.io/Jev_GC/)** —
  install, configuration, API reference, the measured overhead, and an
  explicit account of what those numbers do not prove
- [`docs/quickstart.md`](./docs/quickstart.md)
- [`docs/architecture.md`](./docs/architecture.md)
- [`docs/jev_question_design.md`](./docs/jev_question_design.md)
- [`SPEC.md`](./SPEC.md)

## License

MIT
