Metadata-Version: 2.4
Name: scene-memory
Version: 0.1.0
Summary: Conversational agent memory as versioned slots: knows which version of a fact still holds, and which ones it superseded
Project-URL: Homepage, https://github.com/natanloterio/scene-memory
Project-URL: Repository, https://github.com/natanloterio/scene-memory
Project-URL: Documentation, https://github.com/natanloterio/scene-memory/blob/master/docs/research-journey.md
Project-URL: Issues, https://github.com/natanloterio/scene-memory/issues
Author-email: Natan Loterio <natanloterio@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,conversational-memory,llm,longmemeval,memory,ollama
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Provides-Extra: embed
Requires-Dist: sentence-transformers>=3.0; extra == 'embed'
Provides-Extra: llm
Requires-Dist: anthropic>=0.40; extra == 'llm'
Requires-Dist: openai>=1.0; extra == 'llm'
Provides-Extra: store
Requires-Dist: graphiti-core>=0.3; extra == 'store'
Requires-Dist: mem0ai>=0.1; extra == 'store'
Description-Content-Type: text/markdown

# scene-memory

A **scene layer** for conversational agent memory: turns of conversation become a
graph of versioned **slots** that knows *which version of each fact still holds*.
Ask "where do I live?" and it answers with the current value; ask "where did I
live *before*?" and it answers from the superseded history — nothing is deleted,
facts are invalidated, not forgotten.

Two things live in this repository:

- **The library** (`scene_memory/`) — pure-stdlib core, no `pip install`, no
  network. Extraction and reading use a local LLM through Ollama.
- **The benchmark system** (`results/extractor_experiment/`) — the research
  harness that routes each question by form to the right memory and reader,
  and holds the headline result below.

## Headline results

**LongMemEval-S, official judging protocol: 477/500 = 95.4% — above
[Mastra](https://mastra.ai) (gpt-5-mini) on all six indicators.**

| Indicator | This system | Mastra target | |
|---|---:|---:|---|
| multi-session | 119/133 | 116 | above |
| temporal-reasoning | 128/133 | 127 | above — at the measured oracle ceiling |
| knowledge-update | 76/78 | 75 | above |
| single-session-user | 68/70 | 67 | above |
| single-session-assistant | 56/56 | 53 | above |
| single-session-preference | 30/30 | 30 | ties the benchmark ceiling (the max defined) |

Every number comes from a **single reproducible pass** over the 500 questions
(`sonda_e2e_canonica.py`), verified against a canonical state file whose guard
refuses to write if anything drifts. Every mechanism behind it was integrated
with a **pre-registered prediction committed before measuring**, an explicit
falsification bar, and ten falsified arms published with the same prominence as
the wins. The full audit trail is in
[`results/extractor_experiment/RELATORIO_ARQUITETURAS.md`](results/extractor_experiment/RELATORIO_ARQUITETURAS.md)
(Portuguese).

**And the library result that motivates the scene itself:** reading the
compressed scene (~550 words) lets a small local model answer *better than the
same model reading the raw sessions* (~9,000 words) — +0.102 accuracy at **16×
less input** — matching a much stronger reader on raw text. The scene removes
exactly the noise that confuses readers: duplicates, stale values, scattered
updates.

> Honest framing: these are benchmark results with registered caveats, not a
> product promise. Router triggers and absence gates are regexes calibrated on
> LongMemEval's English corpus; see [Limits](#limits).

## Quickstart

Core is pure stdlib. For extraction/reading you need a local
[Ollama](https://ollama.com) with the model used by the frozen research
protocol:

```bash
ollama pull gemma4:12b
```

```python
from scene_memory import SceneMemory, make_client

client = make_client("ollama:gemma4:12b")
memory = SceneMemory("./cache", client, conversation_id="demo")

# WRITE — assertions become facts; questions never touch the memory
memory.remember("I live in Lisbon and work as a software engineer.")
memory.remember("My cat is called Whiskers.")
memory.remember("Actually, I moved to Porto last month.")

# READ — answers from the scene, not by re-reading the conversation
memory.answer("Where do I live?")          # -> "porto"
memory.answer("Where did I live before?")  # -> "lisbon"  (superseded history)

# PROVENANCE — the answer carries the facts that produced it
answer = memory.answer("Where do I live?")
answer == "porto"                          # it IS a string; old call sites unchanged

for fact in answer.fatos:
    print(fact.indice, fact.slot, fact.valor, fact.corrente)
# 4 user|location porto True
# 5 user|date+move last month True

for alert in answer.alertas:
    print(alert.codigo)
# valor_ausente_no_texto   -- the reader cited fact #5 but "last month" never
#                             reached the answer text. Informational, not an
#                             error: paraphrase legitimately drops values.

# INSPECT — the scene is a plain, walkable structure
for key, slot in memory.scene.slots.items():
    current = slot.current
    print(key, "->", current.display if current else None,
          "| history:", [v.display for v in slot.superseded])
```

What you get back is honest about its own limits, in two ways.

**It shows its work.** `answer()` returns a `Resposta`, which subclasses `str`
— it compares, prints and serialises like the plain string it replaced, so
existing call sites need no change — and carries `.fatos`, the scene facts the
reader said it used, resolved back to concrete slots and values. Each one knows
whether the value it cites is the slot's **current** value or a **superseded**
one. (Note the one trap: string operations like `.strip()` return a plain `str`
and drop the provenance. Keep the object if you need it.)

**And the code checks that work.** `.alertas` carries what the verifier
contested about the citation, structurally, without pretending to judge
semantics: `valor_superado` when a cited value is no longer current,
`indice_invalido` for a fact number that does not exist, `sem_citacao` when the
reader cited nothing. A superseded citation is *not* automatically wrong — it is
the right answer to a question about the past — so the alert says so rather than
accusing.

Measured cost of asking for citations, paired over the 422 held-out questions:
120/422 → 122/422, McNemar p = 0.75. It is free
([`results/portao_citacao/`](results/portao_citacao/RESULTADO.md)).

The lower-level `ask()` still carries `abstained` (the scene had no matching
slot — not the same as "I don't know") and `may_be_partial` (other slots also
matched the question, and single-slot reading returned just one).

### How the scene works, in 30 seconds

Every asserted fact becomes a `(subject, relation, object)` triple filed under
a **canonical key** — `user|location` — so paraphrases land in the same slot
without embeddings. A slot keeps *all* values it ever had, ordered by logical
time; the newest is **current**, the rest are **superseded**:

```
user|location
  t1  lisbon    superseded
  t3  porto     current
```

Updates supersede, enumerations coexist, and questions about the past read the
history. Writing is immutable: each turn produces a new scene, so a failure
mid-turn never leaves memory half-written.

## Live demo — chat with the scene at its side

A minimal chat where you watch every slot being born, updated and superseded,
turn by turn. Under each answer it prints the facts that produced it — fact
number, slot, value, and a **SUPERSEDED** mark when the reader cited a value the
scene has since replaced. The right-hand panel is a full inspector: click a slot
for its complete value history with origin text, filter slots live, see which
slot answered. There is a button to reset the scene without restarting the
server:

```bash
ollama serve                                   # in another terminal
python3 demo/chat/servidor.py                  # http://127.0.0.1:8000
# from another device on your Tailscale network:
python3 demo/chat/servidor.py --host $(tailscale ip -4)
```

The demo is also an instrument: its API-friction findings (24 so far, 15 already
fixed upstream) are logged in [`demo/chat/ACHADOS.md`](demo/chat/ACHADOS.md).
The open ones are worth reading — they are where this system currently lies to
you, written down in the same detail as the wins.

## Reproducing the evaluation

```bash
python3 tests/test_core.py                     # core tests, no LLM
python3 -m pytest tests/ -q                    # full suite (475 tests)

# retrieval baselines (no LLM):
python3 -m scene_memory.cli run --dataset lme_ku_s --retriever bm25 grep vector --k 1 5

# structural scene on real conversation (needs Ollama + gemma4:12b):
python3 -m scene_memory.cli scene-query-conv --dataset lme_ku_s \
    --resolver-model ollama:gemma4:12b --select-model ollama:gemma4:12b
```

Benchmark datasets are **not** shipped in the repository. Fetch scripts live in
`eval/` (`fetch_mab.py` for MemoryAgentBench via HuggingFace; LongMemEval per
its [official instructions](https://github.com/xiaowu0162/LongMemEval) into
`data/lme_raw/`). The benchmark-system caches under
`results/extractor_experiment/` are likewise local-only; the scripts re-extract
on demand.

## Repository layout

| Path | What it is |
|---|---|
| `scene_memory/` | the library: types, extraction, scene assembly, reading, retrieval baselines, eval harness |
| `demo/chat/` | the live demo + API-friction findings (`ACHADOS.md`) |
| `results/extractor_experiment/` | the routed benchmark system, canonical state with guard, single verification pass, and the full campaign report |
| `docs/research-journey.md` | **the paper**: the complete research journey, stage by stage, with every measurement, falsification and retraction ([PT original](docs/research-journey.pt.md)) |
| `docs/journey.html` | the journey as a navigable page (charts + glossary) |
| `tests/` | 475 tests, no network required |
| `CONTRIBUTING.md` | the seven open findings, the frozen-protocol rule, three ways in |

## Limits

Declared, measured, and kept visible — not fine print:

- **Benchmark numbers are benchmark numbers.** The router triggers, absence
  gate and date anchors are regexes calibrated on LongMemEval's English
  corpus; in another language or domain they do not fire without re-measurement.
- **Two indicators sit at their measured oracle ceiling** (temporal at 128,
  multi-session above its own ceiling): further progress there requires a
  stronger reader model, not better retrieval — this is measured, not assumed.
- **Single-slot reading answers one slot.** Questions whose answer spans
  several facts get a partial answer with `may_be_partial=True`; composing
  across slots is the multi-hop path, a separate experiment.
- **Reported speech is an open finding** (demo finding #15): "they *said* there
  was gold" is extracted as a plain fact and the negation is lost as structure.
  The extractor already marks modality informally; making it a contract — and
  making the reader state it — is the mapped next research step.
- **Frozen defaults are the research protocol** (`v4`, `passes=2`,
  `merge_sim=0.5`). Changing them breaks comparability with every published
  measurement, so changing them is an explicit caller decision.

## Research documentation

The journey from a 12-example probe to the full result — including the
pre-registered held-out evaluation where **the central thesis initially
failed** and what was done about it honestly — is the paper:
[`docs/research-journey.md`](docs/research-journey.md)
([Portuguese original](docs/research-journey.pt.md)). The final
benchmark campaign (arms 28–50, six integrated mechanisms, two external
instruments, ten falsifications, one retraction) is in
[`results/extractor_experiment/RELATORIO_ARQUITETURAS.md`](results/extractor_experiment/RELATORIO_ARQUITETURAS.md).
The campaign report is in Portuguese; the commit history narrates the same
story with predictions committed before every measurement.

## Contributing

[`CONTRIBUTING.md`](CONTRIBUTING.md) has the seven open findings in a table with
the shape of the work for each, the frozen-protocol rule, and what a change has
to look like to be believable here. Three ways in, by effort: run the live demo
and report what breaks; take an open finding; or replicate where these numbers
explicitly do not claim to hold — another language, another model, another
domain. A clean negative result gets published as one.

## License

[MIT](LICENSE).
