Metadata-Version: 2.4
Name: hy-memory
Version: 2.0.1
Summary: HY Memory - Industrial-grade dual-system cognitive memory framework
Author: alvinfei
License-Expression: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic
Requires-Dist: numpy
Requires-Dist: openai<3,>=1.40
Requires-Dist: python-dotenv
Requires-Dist: pyyaml
Requires-Dist: chromadb>=1.5.9
Requires-Dist: kuzu
Requires-Dist: langdetect
Requires-Dist: scikit-learn
Requires-Dist: tiktoken
Requires-Dist: json-repair
Requires-Dist: rank-bm25
Requires-Dist: jieba
Provides-Extra: qdrant
Requires-Dist: qdrant-client; extra == "qdrant"
Requires-Dist: fastembed; extra == "qdrant"
Provides-Extra: faiss
Requires-Dist: faiss-cpu; extra == "faiss"
Provides-Extra: graph
Requires-Dist: neo4j; extra == "graph"
Provides-Extra: mysql
Requires-Dist: aiomysql; extra == "mysql"
Provides-Extra: rerank
Requires-Dist: torch; extra == "rerank"
Requires-Dist: transformers; extra == "rerank"
Provides-Extra: all
Requires-Dist: chromadb>=1.5.9; extra == "all"
Requires-Dist: qdrant-client; extra == "all"
Requires-Dist: fastembed; extra == "all"
Requires-Dist: faiss-cpu; extra == "all"
Requires-Dist: kuzu; extra == "all"
Requires-Dist: neo4j; extra == "all"
Requires-Dist: aiomysql; extra == "all"
Requires-Dist: torch; extra == "all"
Requires-Dist: transformers; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: httpx; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# HY Memory

Production-grade dual-system cognitive memory for LLM agents.

English | [中文](README.zh.md)

HY Memory provides three memory scenes (`scene`), one per kind of task:

| `scene` | Tasks | Memory shape |
|---|---|---|
| `chat` | Small talk, personal information, preferences, general Q&A | 7-layer cognitive memory (facts / summaries / profile / schemas / intentions) |
| `working` | Productivity work: documents, spreadsheets, research, financial analysis, content creation, computer operation | Two-layer memory: session facts + reusable experiences |
| `coding` | Software engineering: coding, debugging, testing, builds, APIs/SDKs, scripts, SQL | The same two-layer memory as `working`, plus a code version-evolution chain (diff chain) |

## Repository Layout

This repository is a monorepo for the memory engine and its end-user application:

- `hy_memory/`, `tests/`, and the root `pyproject.toml` contain the HY Memory Python SDK.
- `memhub/` contains the MemHub macOS app, CLI, local Runtime/Core, Web UI, and integrations.
- `plugins/` contains SDK-level integrations maintained with the engine.

The two build systems are separate. Work from the repository root for the SDK, and from `memhub/` for the application:

```bash
# SDK
pip install -e ".[dev]"
pytest tests/

# MemHub
cd memhub
pnpm install
uv sync --directory apps/core --all-groups --locked
pnpm check
```

Licensing is component-specific: the SDK is covered by the root [MIT license](LICENSE), and MemHub by
[`memhub/LICENSE`](memhub/LICENSE) (Apache-2.0). See [`docs/monorepo.md`](docs/monorepo.md) for the import boundary.

---

## Architecture

### 1. Chat: 7-layer cognitive memory

For long-term personal memory. Raw conversations are abstracted layer by layer into facts, summaries, a profile, and higher-level cognitive structures:

![Chat architecture](docs/images/chat_architecture.svg)


| Layer | Name | Content |
|---|---|---|
| L0 | BASIC_INFO | Basic information |
| L1 | RAW | Raw conversation (append-only) |
| L2 | FACT | Atomic facts (versioned, immutable records) |
| L3 | SUMMARY | Session summaries |
| L4 | IDENTITY | Identity profile |
| L5 | KNOWLEDGE | Knowledge graph (entities / relations / topics; not yet implemented) |
| L6 | SCHEMA | Mental models (abstract narrative templates, graph layer) |
| L7 | INTENTION | Prospective intentions (with an expiry; demoted to ordinary facts once expired) |

**Write path (System 1)**

```
add() → embed → store the original text as L1_RAW
      → [pro/ultra] LLM extracts facts, summaries, intentions → reconcile (conflicts and updates) → L2 / L3 / L7
      → once extraction finishes, L1_RAW is demoted to shadow and no longer recalled
      → [ultra] System 2, triggered explicitly by evolve(): scans all memories, induces schemas + detects cross-domain patterns
```

Updates never overwrite old records; they are linked through a `supersedes` chain, so every memory's history can be traced.

**Modes**

| Mode | What it does | Graph DB | Use case |
|------|--------------|----------|----------|
| `lite` | Embedding-only writes, no LLM calls | No | Fast ingestion, zero LLM cost on write |
| `pro` | + LLM extraction + reconcile | Optional (`MEMORY_ENABLE_GRAPH`, off by default) | Standard workloads |
| `ultra` | + System 2 schema induction + cross-domain sweeper (triggered by `evolve()`) | Yes (always on) | Full cognitive architecture |

**Read path**: vector-similarity recall, returned in three channels — `profile`, `normal`, and `proactive` (prospective intentions).
For reader selection, quotas, and rerank, see the [Search and Reader guide](docs/hy_memory_search_skill.md).

### 2. Working: two-layer productivity memory

For getting things done. One session with tool calls (a trace) produces both layers at once (see the diagram in the Coding section below; Working is the same pipeline without the orange parts):

| Layer | What it records | Per session | Structure |
|---|---|---|---|
| **Facts** (WorkingFact) | What this session did: the task, the approach, the result | Exactly 1, no threshold | `statement`, `subject`, `files`, `details` (verbatim source) |
| **Experiences** | Approaches that can be reused next time a similar task comes up | 0–N, must pass a value threshold | `task`, `search_keys`, `solution`, `boundary_envs`, `confidence` |

The fact layer guarantees that work which was done is remembered; the experience layer makes what was learned reusable. The two layers link to each other:
`fact.derived_experience_ids` ↔ `experience.source_fact_ids`.

**Write path**

```
add(messages, scene="working")
  1. Preprocess: trim oversized tool results and assistant text; window history by user turn
  2. One LLM extraction call produces both fact drafts and experience drafts
  3. Experience reconcile: recall similar experiences; the LLM decides ADD / UPDATE / DELETE / SKIP
  4. Fact reconcile: every new fact is inserted; if it continues the same task, the old fact is marked
     SUPERSEDED (soft delete, linked through fact_link into a task-progress chain) and its experiences
     are re-parented to the new fact
  5. When a fact is superseded, a one-paragraph progress summary (fact_chain_summary) is generated and
     returned with the fact at search time
  6. Facts and experiences from the same batch are linked bidirectionally
```

Superseding never loses history: old facts stay in the superseded state and remain traceable, and the progress summary shows how the task got to where it is.

**Read path**

```
search(query, scene="working")
  1. Embed the query (one embedding call, no LLM)
  2. Facts: vector recall over active facts + light lexical rerank, threshold filter, top-k
  3. Experiences: experiences linked from matched facts first, then directly recalled ones, top-k
  4. Return facts and experiences, at most `limit` of each
```

`subject` can optionally act as a hard constraint on superseding (different subjects are never merged). It is enabled with `HY_FACT_SUBJECT_GATE` and off by default.

### 3. Coding: two-layer memory + version-evolution chain

![Coding architecture](docs/images/coding_architecture.svg)

Coding **reuses the same two-layer pipeline and data schema** as Working. It differs in three ways:

| | Working | Coding |
|---|---|---|
| Extraction spec | Productivity-task spec | Software-engineering spec: the subject must be code or a decision; record exact coordinates (file / function / branch) instead of pasting code |
| Storage | `coding_memory.db` + `*_coding_keys` / `*_working_facts` collections | `coding_memory_cv.db` + `*_cv` collections, fully isolated from working |
| Version-evolution chain | None | **Yes**: how many times the same code was changed, and what each change was |

**Why a version-evolution chain**: fact extraction deliberately avoids pasting code (the budget goes to more informative coordinates), so a version that was later reverted never appears in a fact.
That version is usually not in the working tree or at HEAD either, so it cannot be read back afterwards. The chain stores the real diff of every version with zero LLM calls, filling in exactly the information a summary inevitably drops.

**Design** (`hy_memory/coding/chain_store.py`)

- **A decorator, not a retriever**: the vector layer decides which facts are returned; the chain only attaches, under each fact, every revision of the code it touches (the user's words for that revision + the real diff). It does no similarity search, picks no version, and does not decide which version is final.
- **Change sources**: local Git diffs (set `MEMORY_CODING_CHAIN_REPO_ROOT`), or deterministic replace actions with path / old / new in the tool trace. Edits that cannot be parsed reliably are skipped.
- **Chain granularity**: one thread is "one block of one function in one file on one branch"; hunks whose header names a function are grouped by function, others by clustered line ranges, falling back to file granularity.
- **Isolation**: a thread's owner is `user_id` plus `workspace_id`, so patches from different users or repositories never mix.
- **Best-effort**: a chain-recording failure never blocks two-layer memory writes or searches.

Coding search additionally returns:

- `chain_block`: version-evolution context aligned with the recalled facts; `""` when nothing can be attached;
- `chain_used`: indices into `facts` that are already covered by `chain_block`. To build "chain block + remaining facts", filter `facts` by `chain_used`.

---

## SDK Usage

### Install

```bash
pip install hy-memory            # core (includes ChromaDB)
pip install hy-memory[qdrant]    # + Qdrant
pip install hy-memory[faiss]     # + FAISS
pip install hy-memory[graph]     # + Neo4j (Kuzu is a core dependency)
pip install hy-memory[mysql]     # + MySQL cache
pip install hy-memory[rerank]    # + local multilingual cross-encoder rerank
pip install hy-memory[all]       # everything
```

### Configuration

Only an LLM and an embedding endpoint are needed; any OpenAI-compatible service works:

```bash
export MEMORY_LLM_API_KEY="sk-your-key"
export MEMORY_LLM_BASE_URL="https://api.deepseek.com"
export MEMORY_LLM_MODEL="deepseek-chat"

export MEMORY_EMBEDDER_API_KEY="sk-your-key"
export MEMORY_EMBEDDER_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"
export MEMORY_EMBEDDER_MODEL="text-embedding-v3"
export MEMORY_EMBEDDING_DIMS=1024
```

Or use the OpenAI defaults with a single key:

```bash
export OPENAI_API_KEY="sk-your-key"
```

See the [environment variable reference](docs/env_reference.md) for the full list.

### Chat memory

```python
from hy_memory import HyMemoryClient

client = HyMemoryClient(mode="pro")

# Write: plain text or OpenAI-format messages
client.add("I love sci-fi movies, especially Interstellar", user_id="user_1")
client.add([
    {"role": "user", "content": "Recommend a movie"},
    {"role": "assistant", "content": "Try Interstellar — a sci-fi masterpiece by Nolan"},
], user_id="user_1")

# Search: returns the profile / normal / proactive channels
results = client.search("What movies does the user like?", user_ids=["user_1"], limit=10)
for mem in results["memories"]["normal"]:
    print(f"  [{mem['score']:.2f}] {mem['content']}")

client.close()
```

### Working / Coding memory

Writes must include a tool-call trace (`tool_calls` or `role="tool"` messages):

```python
messages = [
    {"role": "user", "content": "Fix the timeout in the search client."},
    {"role": "assistant", "content": "", "tool_calls": [{
        "id": "call_1", "type": "function",
        "function": {"name": "apply_patch", "arguments": '{"path":"src/search.py"}'},
    }]},
    {"role": "tool", "tool_call_id": "call_1", "name": "apply_patch",
     "content": "Patch applied successfully"},
]

client.add(
    messages,
    scene="coding",                              # or "working"
    user_id="user_123",
    workspace_id="github.com/example/service",   # optional: isolate by repository / project
    branch="fix/search-timeout",                 # optional: requires workspace_id
)

result = client.search(
    "Which timeout implementation was finally kept?",
    scene="coding",
    user_ids=["user_123"],
    workspace_id="github.com/example/service",
    branch="fix/search-timeout",
    limit=5,
)

result["facts"]          # fact layer
result["experiences"]    # experience layer
result["chain_block"]    # coding only: version-evolution context
```

### Automatic routing

When `scene` is omitted:

- **Writes**: input without tool calls goes straight to chat with no LLM call; input with tool calls is classified once by an LLM into chat / working / coding, falling back to chat on failure.
- **Searches**: a single query with `search_mode="default"` always goes to chat with zero LLM calls; `search_mode="agentic"` performs scene classification, query rewriting, and time-range parsing.
- An explicit `scene` goes straight to that path without calling the classifier.

See [Working/Coding Scenes](docs/scene_coding_usage.md) for the full routing rules, response shapes, and storage layout.

### Separate memories by source

`source` identifies the application that produced a write. It is independent of `agent_id`, so OpenClaw, Cursor, and Codex can write for the same user and agent without conflating caller identity with agent identity.

```python
client.add("...", user_id="u1", source="openclaw")
client.add("...", user_id="u1", source="cursor")
client.add("...", user_id="u1", source="codex")

result = client.search(
    "How was this implemented?",
    user_ids=["u1"],
    include_sources=["codex", "openclaw"],
    exclude_sources=["cursor"],
)
```

Source values are normalized with `strip().lower()`. Include and exclude filters are combined in one backend query, with exclusion taking precedence.
Memories derived from evidence belonging to multiple sources use `source="mixed"`; when any source filter is active, mixed memories are excluded unless `"mixed"` is explicitly included.

### Other APIs

```python
client.get("memory_id")
client.update("memory_id", "updated content")
client.delete("memory_id")
client.list_memories(user_id="u1")

# ultra mode: run System 2 cognitive processing explicitly (blocks until done)
client.evolve(user_id="u1")
```

### Per-call embedding endpoint override

`add()` / `search()` accept an optional `embed_override` that replaces only the embedding **endpoint** for that call
(`base_url` / `api_key` / `extra_headers` / `timeout`). The model and dimensions always come from the client configuration, so vectors stay compatible with what is already stored.

```python
override = {"base_url": "http://localhost:8000/v1", "api_key": "sk-..."}
client.add("The user plays basketball", user_id="u1", embed_override=override)
client.search("hobbies", user_ids=["u1"], embed_override=override)
```

The same capability is exported as a context manager; it is concurrency-safe (contextvar-scoped, reset automatically):

```python
from hy_memory import embed_endpoint_override

with embed_endpoint_override(base_url="http://localhost:8000/v1", api_key="sk-..."):
    ...  # every embed call inside the block uses the override endpoint
```

---

## Storage Backends

- Vector stores: ChromaDB (default), Qdrant, FAISS
- Graph stores (ultra mode): Kuzu (default), Neo4j
- Working / Coding metadata: local SQLite

## Documentation

- [Usage guide](docs/usage.md) — full configuration reference, API details, deployment modes
- [Environment variables](docs/env_reference.md) — settings and defaults
- [Search and Reader guide](docs/hy_memory_search_skill.md) — reader selection, schema quotas, rerank, and return channels
- [Working/Coding Scenes](docs/scene_coding_usage.md) — productivity-memory routing, response shapes, and coding chain behavior
- [Graph memory design](docs/pro_graph_design.md) — graph projection, rerank, relation paths, and deployment limits
- [Environment template](.env.example) — a copy-ready configuration template
- [OpenClaw plugin](plugins/openclaw/README.md) — agent integrations live under `plugins/` (overview: [plugins/README.md](plugins/README.md))

## License

SDK: MIT. MemHub: Apache-2.0.
