Metadata-Version: 2.4
Name: harness-memory
Version: 0.1.3
Summary: Pluggable memory system with hierarchical recall, FTS search, and multiple backend support.
Project-URL: Homepage, https://github.com/orcakit/harness-memory
Project-URL: Repository, https://github.com/orcakit/harness-memory
Project-URL: Issues, https://github.com/orcakit/harness-memory/issues
Author: orcakit
License-Expression: MIT
License-File: LICENSE
Keywords: agent,fts,llm,memory,recall,sqlite
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Provides-Extra: chroma
Requires-Dist: chromadb>=0.4; extra == 'chroma'
Provides-Extra: cli
Requires-Dist: click>=8.0; extra == 'cli'
Provides-Extra: dev
Requires-Dist: click>=8.0; extra == 'dev'
Requires-Dist: langgraph-checkpoint-sqlite>=2.0; extra == 'dev'
Requires-Dist: langgraph-checkpoint>=2.0; extra == 'dev'
Requires-Dist: langgraph>=0.2; extra == 'dev'
Requires-Dist: mypy>=1.13; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=2.2; extra == 'embeddings'
Provides-Extra: langgraph
Requires-Dist: langgraph-checkpoint-sqlite>=2.0; extra == 'langgraph'
Requires-Dist: langgraph-checkpoint>=2.0; extra == 'langgraph'
Provides-Extra: langgraph-postgres
Requires-Dist: langgraph-checkpoint-postgres>=2.0; extra == 'langgraph-postgres'
Requires-Dist: langgraph-checkpoint>=2.0; extra == 'langgraph-postgres'
Requires-Dist: psycopg[binary]>=3.1; extra == 'langgraph-postgres'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.7; extra == 'qdrant'
Description-Content-Type: text/markdown

<p align="center">
  <img src="assets/images/banner.jpeg" alt="Harness Memory Banner" width="100%" />
</p>

<h1 align="center">Harness Memory</h1>

<p align="center">
  <strong>Pluggable long-term memory system for LLM agents — structured recall, full-text search, and automatic knowledge extraction from conversations.</strong>
</p>

<p align="center">
  <a href="https://pypi.org/project/harness-memory/"><img src="https://img.shields.io/pypi/v/harness-memory.svg" alt="PyPI version" /></a>
  <a href="https://github.com/orcakit/harness-memory/actions/workflows/ci.yml"><img src="https://github.com/orcakit/harness-memory/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
  <a href="https://pypi.org/project/harness-memory/"><img src="https://img.shields.io/pypi/pyversions/harness-memory.svg" alt="Python versions" /></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT" /></a>
</p>

<p align="center">
  <b>English</b> · <a href="README_CN.md">中文</a>
</p>

---

## Highlights

- **Zero Dependencies** — Core package uses only Python stdlib + sqlite3; no heavy ML frameworks required
- **L0→L4 Memory Pipeline** — Automatic extraction of facts from conversations, promotion to structured atoms, entity pages, and audit journal
- **Multi-Stage Recall** — Query parsing, routing, multi-source gather, rerank, diversify, suppress, token budget, cache, and co-reference resolution
- **Hierarchical Memory Tree** — Manual root → branch → leaf structure; maintained independently and included in recall
- **Pluggable Backends** — SQLite (default), PostgreSQL
- **Host Integrations** — Native plugins for OpenClaw and Hermes agent platforms
- **Full CLI** — 20+ command groups for every layer of the system

---

## How It Works

Harness Memory runs a five-layer pipeline that converts raw conversation events into structured, searchable knowledge:

```
L0  RawEvent       Host messages/events captured according to adapter configuration (immutable)
      │  LLM extraction (agent_end or scheduled)
      ▼
L1  Candidate      LLM-extracted fact awaiting promotion (Fact/Decision/Task/Preference)
      │  5-check promotion worker (value → evidence → entity → duplicate → conflict)
      ▼
L2  AtomCard       Structured fact linked to an entity, FTS-indexed, append-only
      │  grouped by
      ▼
L3  Entity         Anchor node (User/Person/Project/…) + alias table
    EntityPage     LLM-generated markdown summary, regenerated when marked dirty
      │  decisions and lifecycle changes logged to
      ▼
L4  Journal        Append-only audit log (promote/merge/conflict/deprecate/gc)
```

At recall time, a query runs through the **recall pipeline** (parse → route → gather → rerank → diversify → suppress → budget) and returns snippets from atoms and raw events. The hierarchical tree remains available as an organizational view: root/branch nodes store labels, while each leaf references one canonical `AtomCard`. `EntityPage` records are generated and readable through the page APIs, but page headlines are not yet a source in the main recall gather path.

---

## Quick Start

### Installation

```bash
pip install harness-memory
```

With CLI support:

```bash
pip install "harness-memory[cli]"
```

Optional backends:

```bash
pip install "harness-memory[postgres]"   # PostgreSQL backend
```

### Python API

```python
from harness_memory import Memory
from harness_memory.recall import recall_for_prompt_v2

memory = Memory(namespace="my-agent")

# Store a fact; this creates RawEvent → Candidate → AtomCard plus a leaf reference
memory.store("User prefers Python over Java", topic="preferences")

# Recall via the full M4 pipeline (atom + raw, reranked)
result = recall_for_prompt_v2(memory, "programming language preference")
print(result.rendered)   # markdown block ready for prompt injection

# Or recall canonical atoms projected as MemoryNode leaves
for node in memory.recall("programming language"):
    print(f"[{node.topic}] {node.content}")
```

### Auto-extraction from conversations

```python
from harness_memory import Memory
from harness_memory.extractor import CandidateExtractor
from harness_memory.llm import LLMClient  # plug in your host LLM

memory = Memory(namespace="my-agent")
extractor = CandidateExtractor(llm=my_llm_client)

# At session end: extract candidates from L0 raw events
from harness_memory.extractor import extract_session
result = extract_session(memory=memory, extractor=extractor, session_id="sess-123")

# Promote pending candidates to structured atoms
promotion = memory.promote_candidates()
print(f"promoted={promotion.promoted}, merged={promotion.merged}, conflicts={promotion.conflicts}")
```

### CLI

```bash
# Ingest conversation files
harness-memory --namespace my-agent ingest --source ~/.claude/projects/myapp/

# Recall
harness-memory --namespace my-agent recall "user authentication preferences"

# Inspect the pipeline
harness-memory --namespace my-agent candidate list --status pending
harness-memory --namespace my-agent atom list --entity-id <id>
harness-memory --namespace my-agent entity list
harness-memory --namespace my-agent memory tree

# Operations
harness-memory --namespace my-agent export --out backup.jsonl
harness-memory --namespace my-agent gc run
```

---

## Architecture

### Storage Layers

| Layer | Type | Table(s) | Description |
|---|---|---|---|
| L0 | `RawEvent` | `raw_events` | Immutable evidence log |
| L1 | `Candidate` | `candidates` | LLM-extracted facts awaiting promotion |
| L2 | `AtomCard` | `atoms` | Promoted structured facts; primary recall source |
| L3 | `Entity`, `EntityPage`, `Alias` | `entities`, `entity_pages`, `aliases` | Entity anchors + auto-generated summaries |
| L4 | `JournalEntry` | `journal` | Append-only audit log |
| — | `MemoryNode` | `memory_nodes` | Hierarchical organization; leaves reference canonical atoms |

### Promotion Pipeline (L1 → L2)

The promotion worker runs 5 checks in order. First terminal result wins:

1. **Value** — drop if `importance=low AND confidence=low` (chit-chat noise)
2. **Evidence** — verify cited raw event IDs exist in L0
3. **Entity** — resolve or create the entity this fact belongs to (alias → name → LLM → new)
4. **Duplicate** — merge if an identical assertion already exists on the entity
5. **Conflict** — flag if polarity flips vs an existing atom (negation detection)

### Recall Pipeline (M4)

`recall_for_prompt_v2` runs the following stages per query:

```
cache lookup → parse query → route (entity/time/coref hints)
→ gather (atom+raw)
→ rerank (BM25 + importance + confidence + recency + layer_prior)
→ diversify (≤N per entity) → suppress (Jaccard dedup)
→ token budget enforcement → active-entity update
→ render markdown → cache result
```

### Memory Tree ↔ Canonical Atoms

The `MemoryNode` tree keeps its original root → branch → leaf model without creating a second fact store:

- **Root / branch**: store organizational labels in `MemoryNode.content`.
- **Leaf**: stores a unique `atom_id`; backends project `AtomCard.assertion` into `MemoryNode.content` when reading.
- **Manual write**: `Memory.store(..., level="leaf")` creates `RawEvent → Candidate → AtomCard`, then links the leaf.
- **Promotion / direct atom write**: `Memory.add_atom()` creates the same leaf reference automatically.
- **Recall**: `Memory.recall()` searches atoms and returns their linked leaves; `recall_for_prompt_v2()` gathers atom + raw only.

Leaf facts are append-only through `AtomCard`: `Memory.update(..., content=...)` rejects leaf text changes, while deleting a leaf deprecates its atom. This preserves the tree API and removes the previous two-source-of-truth ambiguity.

---

## CLI Reference

```
harness-memory [GLOBAL OPTIONS] COMMAND [ARGS]
```

**Global options:** `--config PATH`, `--backend sqlite|postgres|qdrant`, `--db PATH`, `--dsn DSN`, `--namespace NS`, `--json`

The CLI still accepts `qdrant` as a reserved backend choice for compatibility, but this repository does not currently ship a `QdrantMemoryBackend`.

| Command group | Description |
|---|---|
| `config show/set` | Show or set configuration values |
| `ingest` | Archive conversation files (Claude Code, OpenAI, generic JSONL) |
| `conversations list/get/export` | Manage stored conversations |
| `summary pending/set` | Manage conversation summaries |
| `raw add/list/search/show` | L0 raw event management |
| `candidate extract/list/show/promote/review/fallback` | L1 extraction, promotion, inspection, and manual review |
| `atom list/show/search` | L2 atom inspection |
| `entity list/show` | L3 entity management |
| `page show/list-dirty/regen/edit` | L3 entity page management |
| `journal list` | L4 audit log |
| `memory store/get/update/delete/tree/recall` | MemoryNode tree management |
| `recall` | Run the full M4 recall pipeline |
| `thread show` | Inspect LangGraph checkpoint thread state |
| `export / import / migrate / backfill` | Migration and data operations |
| `gc run`, `consolidate run` | Lifecycle maintenance |
| `openclaw setup/doctor/uninstall/print-config` | OpenClaw host integration |
| `hermes install/doctor` | Hermes host integration |

---

## Host Integrations

Harness Memory can run as a native memory provider inside agent hosting platforms.

### OpenClaw

```bash
pip install "harness-memory[cli]"
openclaw plugins install npm:@agentmemory/openclaw-plugin
harness-memory openclaw setup    # register the plugin and bridge Python
harness-memory openclaw doctor   # verify the integration
```

The OpenClaw plugin (`plugins/openclaw/agentmemory/`) is a TypeScript shell that spawns the Python bridge (`agentmemory-bridge`) and registers `registerMemoryCapability` with the host runtime.

### Hermes

```bash
pip install harness-memory-hermes
harness-memory-hermes install    # install the Hermes adapter
harness-memory-hermes doctor     # verify the integration
```

See `docs/release-packaging.md` for release artifact boundaries and `docs/agentmemory-openclaw-hermes-usage.md` for source-checkout setup guides.

---

## Python API Reference

### Memory

```python
from harness_memory import Memory

memory = Memory(namespace="my-agent", backend="sqlite",
                backend_config={"db_path": "~/my-memory.db"})
```

| Method | Layer | Description |
|---|---|---|
| `store(content, topic, level, parent_id, metadata)` | Tree/L0-L2 | Create a directory node, or create an atom-backed leaf |
| `get(node_id)` | Tree | Fetch a MemoryNode by ID |
| `update(node_id, content, topic, metadata)` | Tree | Update directory content or leaf organization; leaf fact text is immutable |
| `delete(node_id, cascade)` | Tree/L2 | Delete organization and deprecate linked atoms |
| `recall(query, limit)` | L2→Tree | Search atoms and return linked MemoryNode leaves |
| `get_tree()` | Tree | Return the full MemoryNode tree |
| `add_raw(content, event_type, ...)` | L0 | Append a raw event |
| `get_raw(event_id)` / `list_raw(...)` / `search_raw(query, limit)` | L0 | Read or search raw events |
| `add_candidate(candidate)` | L1 | Persist a candidate |
| `get_candidate(id)` / `list_candidates(...)` / `search_candidates(...)` | L1 | Read or search candidates |
| `promote_candidates(candidates, limit, llm_hook)` | L1→L2 | Run promotion worker |
| `add_atom(atom)` | L2 | Persist an atom directly |
| `get_atom(id)` / `list_atoms(...)` / `search_atoms(query, limit)` | L2 | Read or search atoms |
| `add_entity(entity)` | L3 | Persist an entity |
| `get_entity(id)` / `list_entities(...)` | L3 | Read entities |
| `upsert_entity_page(page)` | L3 | Upsert an entity page |
| `get_entity_page(entity_id)` / `list_dirty_entity_pages(...)` | L3 | Read entity pages |
| `append_journal(entry)` | L4 | Append an audit entry |
| `list_journal(...)` | L4 | Query the audit log |
| `add_conversation(record)` | Conv | Persist a conversation |
| `search(query, limit)` | Conv | FTS over conversation messages |
| `list_conversations(...)` | Conv | List conversation summaries |
| `get_conversation(conversation_id)` | Conv | Fetch a complete conversation |

`recall()` and `search()` are intentionally different: `recall()` searches canonical atoms and returns their tree leaves, while `search()` searches archived conversation messages. Use `recall_for_prompt_v2()` below for the atom/raw prompt-recall pipeline with routing, reranking, deduplication, and budgeting.

### Recall

```python
from harness_memory.recall import recall_for_prompt_v2

result = recall_for_prompt_v2(
    memory,
    query="authentication flow",
    thread_id="thread-abc",   # enables co-reference resolution
    limit=5,
    total_budget_tokens=1500,
)
print(result.rendered)        # inject directly into prompt
for snippet in result.snippets:
    print(snippet.layer, snippet.text)   # "atom" | "raw"
```

---

## Supported Conversation Formats

| Format | Extension | Detection |
|---|---|---|
| Claude Code | `.jsonl` | `type` field: `human_message` / `assistant_message` |
| Generic JSONL | `.jsonl` | `role` + `content` fields per line |
| OpenAI JSON | `.json` | `messages` array with role/content objects |

---

## Development

```bash
make dev          # Install dev dependencies
make lint         # Ruff check + format check
make typecheck    # mypy strict
make test         # pytest
make all          # lint + typecheck + test
```

---

## Related Projects

| Project | Description |
|---|---|
| [harness-agent](https://github.com/orcakit/harness-agent) | Production-grade AI agent platform built on LangChain Deep Agents |
| [harness-browser](https://github.com/orcakit/harness-browser) | AI-friendly browser automation via CDP |
| [harness-im-bridge](https://github.com/orcakit/harness-im-bridge) | Multi-platform IM channel bridge for AI agents |

---

## License

[MIT](LICENSE)
