Metadata-Version: 2.4
Name: memory-lab-mcp
Version: 0.1.1
Summary: Local-first Agent Knowledge curation and evidence index
Author: Harvey Chan
License-Expression: MIT
Project-URL: Homepage, https://github.com/haru3613/memory-lab
Project-URL: Source, https://github.com/haru3613/memory-lab
Project-URL: Issues, https://github.com/haru3613/memory-lab/issues
Keywords: memory,mcp,agents,claude,codex,retrieval
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp>=1.2
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# Memory Lab

Local-first Agent Knowledge curation and evidence search for Claude Code, Codex, OpenClaw, and Hermes.

Obsidian Markdown under `Agent Knowledge/` is the curated source of truth. Memory Lab's SQLite evidence index is disposable and rebuildable; its legacy memory tables remain available for rollback and inspection during migration.

Memory Lab stores no provider API key. Automatic curation invokes the
already-authenticated agent host CLI.

## Operating model

Keep these layers separate:

- **Capture Notes** — append-only Markdown preferences, decisions, lessons, and
  references; the authoritative durable record.
- **Wiki Projection** — rebuildable entity, concept, and source-index pages with
  evidence-backed wikilinks.
- **SQLite** — a derived search index and legacy rollback surface.

Every new memory belongs to exactly one primary entity: `USER`, `AGENT`,
`APP`, or `RUN`. Its semantic kind and writer remain separate fields. Existing
project memory maps to `APP`; Obsidian Markdown remains authoritative.

Agents can add durable knowledge through the append-only `memory_remember` MCP
tool, the explicit `promote` CLI workflow, or an installed lifecycle hook whose
isolated host-model curator produces a validated candidate and indexed receipt.

## Install

```bash
# Stable release from PyPI:
uv tool install memory-lab-mcp

# Today, and for tracking main (needs git credentials while the repo is private):
uv tool install git+https://github.com/haru3613/memory-lab@main
```

Then check it:

```bash
memory-lab --version
memory-lab stats     # prints the store, config file and corpora it resolved
```

`pipx install memory-lab-mcp` works the same way. The installed commands remain
`memory-lab` and `claude-memory-lab`, so host configs do not depend on the PyPI
distribution name.

### Update

For installations already using the PyPI distribution:

```bash
uv tool upgrade memory-lab-mcp
```

Nothing else. Host configs are written with an absolute path to the installed
entry point, so an upgrade swaps the code behind a path that does not move —
no re-wiring, no re-running setup.

The unpublished development builds used `memory-lab` as their distribution
name. Migrate those in this order so two distributions never own the same
commands or Python package:

```bash
uv tool uninstall memory-lab
uv tool install memory-lab-mcp
```

One caveat: setup records the entry point beside the interpreter that ran it.
Install with `uv tool` or `pipx` and that path is stable. Run setup from a
project venv instead and the hook points into that venv, so recreating it
means re-running `setup-hooks`.

Upgrading from the still older `claude-memory-lab` distribution? Uninstall it
before installing `memory-lab-mcp` for the same reason.

### Configure

Corpora are opt-in. An unconfigured install indexes nothing, which is the
privacy default rather than a bug. Create `~/.config/memory-lab/config.toml`:

```toml
[projects]
my-project = "~/.claude/projects/-Users-me-code-my-project"

# Optional. Found automatically on macOS with iCloud.
[obsidian]
vault = "~/path/to/your/vault"
```

Only the `Agent Knowledge/` folder of a vault is ever read. That boundary is
not configurable.

### Wire up your agents

Claude Code and Codex, in one command:

```bash
memory-lab init --dry-run    # prints the plan, changes nothing
memory-lab init
memory-lab doctor            # exits non-zero on any problem
```

`init` refuses rather than writing a command it could not prove runnable, so a
successful run means the hooks work — including from a shell with no PATH. It
names every file it changed and where it backed each one up. `--uninstall`
reverses it.

OpenClaw and Hermes load a plugin bundle instead of hooks, so they install
through their own plugin channels:

```bash
openclaw plugins install memory-lab --marketplace https://github.com/haru3613/memory-lab
hermes plugins install haru3613/memory-lab-hermes --enable && hermes memory setup memory-lab
```

Hermes also needs the package inside its own interpreter — it loads the
provider in-process:
`uv pip install --python ~/.hermes/hermes-agent/venv/bin/python memory-lab-mcp`.
If that interpreter has the unpublished `memory-lab` distribution, remove it
before this install; see the migration command in the wiring guide.

`memory-lab doctor` checks all four, including whether a plugin would open a
different database from the one your hooks write to. Details and the reasoning
behind each step: [docs/wiring](docs/wiring/README.md).

### Develop

```bash
python3.11 -m venv .venv && .venv/bin/pip install -e ".[dev]"
.venv/bin/python -m pytest
```

Setup merges four owned lifecycle hooks into `~/.codex/hooks.json` or
`~/.claude/settings.json` without replacing unrelated hooks. Repeating setup is
safe; remove only Memory Lab's entries with the matching
`setup-hooks --host <host> --uninstall`.

Codex requires a one-time human trust decision before user hooks execute. After
setup, open Codex, run `/hooks`, review the four commands marked
`MEMORY_LAB_HOOK=1`, and approve them. User hooks run outside the sandbox;
Memory Lab does not write Codex's private trust hashes or install a permanent
trust bypass.

The automatic loop is:

- `SessionStart`: retry pending durable captures and repair the Wiki Projection.
- `UserPromptSubmit`: resolve the active USER, AGENT, APP, and RUN identities,
  search their exact union, and inject relevant context.
- `Stop`: persist a bounded, secret-redacted user/assistant transcript capture,
  run an isolated host-model curator, and write
  only 0–3 validated preferences, decisions, lessons, or references through
  `memory_remember`, each assigned to one primary entity, then rebuild the Wiki
  Projection.
- `PreCompact`: Claude Code captures before context compaction; both hosts retry
  pending captures and rebuild the Wiki Projection.

Tool output is excluded from captures. Obsidian is updated only when
`memory_remember` returns an indexed receipt; failures remain pending in SQLite
and retry on a later lifecycle event. An unrecognized Git repository with an
origin remote is onboarded automatically; unsafe or unidentifiable directories
receive explicit manual guidance instead of silently skipping the memory loop.
Claude's curator runs in safe mode with tools and session persistence disabled.

## Obsidian vault contract

Memory Lab traverses exactly one top-level folder in the existing vault:

```text
Agent Knowledge/
├── _Index.md
├── Users/
├── Agents/
├── Runs/
├── Projects/
├── Domains/
├── Playbooks/
├── References/
├── Templates/
└── Wiki/             # generated; safe to rebuild
    ├── Entities/
    ├── Concepts/
    └── Synthesis/    # compatibility path for generated source indexes
```

Everything else is outside the shared agent index, including `Journal/`, `Private/`, `Inbox/`, `Archive/`, `Side Projects/`, `Strategies/`, and `Backtests/`.

A Capture Note should cover one durable claim and uses the versioned v1
contract below. The opaque `id` stays stable if the file moves. Optional
`source_refs` are a flat list of durable provenance strings.

```yaml
---
schema_version: 1
id: mem-12345678-1234-4abc-8def-1234567890ab
type: decision
project: memory-lab
entity_type: app
entity_id: memory-lab
writer_agent: codex
status: active
# Optional on a successor note:
# supersedes: mem-00000000-0000-4000-8000-000000000000
created: 2026-07-26T00:00:00+00:00
source_refs:
  - "issue:#53"
---
```

The shared parser, validator, and renderer govern MCP writes, ingest, and Wiki
compilation. A malformed v1 note fails before derived-state mutation. Existing
legacy active Capture Notes remain readable and are not rewritten or assigned
synthetic IDs.

The canonical lifecycle statuses are `active`, `superseded`, and `retracted`.
To replace a claim, append a new Capture Note with `supersedes` set to the
predecessor's stable ID; the predecessor file stays unchanged. The graph makes
the predecessor effectively `superseded`. To withdraw a claim without a
replacement, append a `retracted` note that supersedes the current head.
Lifecycle links must stay in the same entity scope and memory type. Dangling
links, self-links, cycles, and forks fail before SQLite or Wiki mutation.
Recall and Wiki source indexes expose only effective `active` heads. Superseded and
retracted notes remain in raw evidence, audit history, and the Wiki History
section. Rebuild derives the same lifecycle state from Markdown.

The vault defaults to:

```text
~/Library/Mobile Documents/iCloud~md~obsidian/Documents/
```

Override it with `MEMORY_LAB_OBSIDIAN_DIR` for fixtures or another vault.

Obsidian refreshes Markdown changed by external tools, and iCloud handles device-to-device vault sync. Memory Lab therefore does not run a file watcher or bidirectional sync daemon.

## Compile the Wiki Projection

Preview the deterministic projection without writing:

```bash
memory-lab wiki compile
```

Apply it and refresh the SQLite evidence index:

```bash
memory-lab wiki compile --apply
memory-lab wiki lint
```

The compiler never edits Capture Notes. It creates scope pages with Current and
History sections, deterministic source indexes of current claims, and
concept pages from explicit repo, workflow, tool, and ticket mentions. The
projection records concept occurrences, explicit scope membership, lifecycle
history, and source citations. It does not infer a typed relationship from
concepts in the same note or from retrieval similarity. Re-running compilation
is idempotent, removes only stale pages marked `generated_by: memory-lab-wiki`,
and refuses symlinked or unowned output paths.

## Onboard a project

Run onboarding once from a Git repository:

```bash
memory-lab onboard --repo /path/to/your/repo \
  --title CardDex --alias cardex-template
```

The command derives the canonical project ID from the `origin` remote (for example, `haru3613-carddex`), creates `Agent Knowledge/Projects/carddex/Overview.md` without overwriting an existing note, updates `~/.mem0/project_map.json`, and incrementally indexes the vault. Use `MEMORY_LAB_MEM0_PROJECT_MAP` to override the map path.

## Promote curated knowledge

Promotion is explicit and dry-run by default:

```bash
memory-lab promote --project haru3613-carddex --kind decision \
  --title "Use source-native prices" --from-file /tmp/decision.md
```

Review the rendered note, then repeat with `--apply`. The command writes a
dated APP note under the onboarded project's `Preferences/`, `Decisions/`,
`Lessons/`, or `References/` folder, refuses to overwrite an existing file, and
refreshes the derived SQLite index.

## Ingest Obsidian evidence

Incrementally ingest new or changed Markdown notes:

```bash
memory-lab ingest-obsidian
```

Normal ingestion skips unchanged files and deliberately does not prune missing notes. A partial iCloud sync therefore cannot wipe previously indexed evidence.

When changing the trust boundary or intentionally rebuilding the disposable Obsidian corpus, use the explicit rebuild mode:

```bash
memory-lab ingest-obsidian --rebuild
```

Rebuild preflights every approved note before opening the replacement transaction, then replaces only `project_key='obsidian'` evidence. The memory layer is preserved: links to retained note paths are remapped to their new evidence rows, while provenance for removed paths is retained in memory metadata. If an existing corpus is populated but fewer approved notes are available than `MEMORY_LAB_OBSIDIAN_REBUILD_MIN_NOTES` (default `1`), the rebuild aborts before deleting anything.

Only `.md` files are imported. Allowlisted-root symlinks and nested paths that resolve outside `Agent Knowledge/` fail closed. Empty notes also fail before writes. Searchable text passes through secret redaction before FTS indexing, but credentials and private material still belong outside `Agent Knowledge/`.

## Ingest Claude conversation evidence

```bash
memory-lab ingest --all --incremental --index-entities --extract-candidates

memory-lab stats
```

Claude ingestion is restricted to the corpora listed under `[projects]` in
`~/.config/memory-lab/config.toml`. An unconfigured install ingests nothing.

## Legacy auto-memory import

This migration-only command imports selected one-fact Markdown files into the legacy memory tables. It is retained for rollback compatibility, not for new cross-agent knowledge; use `onboard` and `promote` for that.

Review a read-only plan first:

```bash
memory-lab import-auto-memory \
  --dir ~/.claude/projects/<project>/memory \
  --project cardex-template \
  --include feedback_example.md \
  --include reference_example.md \
  --dry-run
```

Rerun the same command without `--dry-run` to write. `MEMORY.md` is always skipped. The importer accepts scalar `name`, `description`, `type`, and `originSessionId` fields, with `type` and `originSessionId` also allowed one level under `metadata`; folded or otherwise complex YAML values fail preflight. It maps supported Claude memory types into Memory Lab types, stores absolute source-path and content-hash provenance, skips exact duplicates, and supersedes a prior import when the same source changes. It never modifies source files or scans an entire directory automatically. When `MEMORY_LAB_REQUIRE_REVIEW=1`, a batch where one pending source update depends on another pending source's old content fails preflight; split that dependency into separately reviewed imports.

## Search

```bash
memory-lab search "Mission Control no auto-merge" \
  --project mission-control --top-k 5 --debug

memory-lab search "設計決策" \
  --project obsidian --top-k 5
```

ASCII evidence uses SQLite FTS5/BM25. CJK terms use a scoped escaped substring fallback, with BM25 matches ranked ahead of fallback-only results. Every evidence result includes its source path.

An unscoped or Obsidian-scoped CLI search refreshes changed Markdown before querying. If refresh fails, it warns and searches the last valid index. MCP search follows the same lazy-refresh behavior when the requested evidence scope includes Obsidian.

## MCP surface

Agents use this server for retrieval and append-only Agent Knowledge curation:

```bash
memory-lab serve-mcp --agent claude-code
```

- `memory_search(query=..., project="carddex")` searches legacy memory plus CardDex's scoped Obsidian evidence.
- `memory_search(query=..., user_id="harvey", agent_id="codex", app_id="haru3613-carddex", run_id=...)` searches the exact union of active entities.
- `memory_search(query=..., project="obsidian")` searches all approved Obsidian evidence.
- `memory_remember(project="carddex", kind="decision", title=..., content=..., source_refs=[...])` creates a new Markdown note without modifying existing notes.
- `memory_remember(entity_type="user", entity_id="harvey", kind="preference", title=..., content=..., source_refs=[...])` writes user-scoped knowledge.

Evidence is enabled by default. For onboarded projects, the server resolves the folder, canonical ID, and aliases from `Projects/<folder>/Overview.md`; pass `include_evidence=false` only when a caller explicitly wants legacy memories alone.

Search responses include a `query_id` and `index_status`; memory results retain
their integer `memory_id` and expose nullable `capture_id`. Remember responses
include the stable `capture_id`, primary entity, note path, SHA-256, indexed
evidence event ID, derived integer memory ID, and `indexed` status. Identical remember retries are
safe and return `status="existing"`; same-title requests with different content
still fail instead of overwriting the note.

After upgrading an older index, run one successful search or `ingest-obsidian` while the vault is available to backfill canonical IDs and aliases. Until then, an offline legacy index can route project evidence only by folder name.

Use `memory_remember` for explicit requests to remember something and for durable decisions, lessons, and references established during a task. Do not store transient status, speculation, or secrets. Existing notes remain immutable through MCP: update and delete tools are not exposed.

Agent hosts must enforce the start, running, and closeout behavior in the
[session lifecycle contract](docs/wiring/session-lifecycle.md); starting the MCP
process alone does not prove that an agent read or wrote memory.

Do not share one SQLite store across personal and company trust boundaries. Use a separate `--db` path for each boundary; see [`docs/wiring/store-isolation.md`](docs/wiring/store-isolation.md). Per-agent setup lives under [`docs/wiring/`](docs/wiring/).

## Evaluate retrieval

```bash
# evidence layer (raw notes)
memory-lab eval run --question-set baseline

# memory layer: each question asked in exact English, English paraphrase and
# Chinese, plus hard negatives that must return nothing
memory-lab eval run --question-set memory-v2
```

The memory set reports recall@1/@k and MRR per phrasing, a hard-negative
false-positive rate, and the same two numbers for the *injected* set — what the
`UserPromptSubmit` hook would actually have shown an agent. Run it against a
copy of the store if you care about the query log: the eval itself never writes
to `query_log`, but nothing else about `--db` is read-only.

The injection gate keeps hits within `35%` of the top score and above an
absolute floor of `K * log10(active memories)`, disabled below 50 memories.
`K` defaults to `3.0`, calibrated against `eval/memory-v2.json`; override it with
`MEMORY_LAB_INJECT_SCORE_FLOOR_K` and re-run the memory eval to see the effect on
injected recall and the injected false-positive rate.

Generated SQLite databases and eval reports live under gitignored `data/`.

## Read-only UI

The Nuxt 3 viewer under `ui/` browses memories, evidence, and stats. Its SQLite driver is read-only.

```bash
cd ui
npm install
npm run dev
```

It defaults to `../data/claude-memory-lab.sqlite`, which is the *legacy* store
location. The CLI may resolve a different one on a fresh install, so point the
viewer explicitly at whatever `memory-lab stats` reports:

```bash
MEMORY_LAB_DB=/path/to/store.sqlite npm run dev
```

Legacy review actions remain CLI-only for rollback/admin use.
