Metadata-Version: 2.5
Name: memall-os
Version: 0.1.70
Summary: Multi-agent collaborative memory OS
Project-URL: Homepage, https://github.com/j19800/MemALL
Project-URL: Repository, https://github.com/j19800/MemALL
Project-URL: BugTracker, https://github.com/j19800/MemALL/issues
Project-URL: Changelog, https://github.com/j19800/MemALL/releases
Author: MemALL Contributors
License: MIT
License-File: LICENSE
Keywords: agent-memory,ai-agents,knowledge-base,llm,mcp,memory,multi-agent,semantic-search,vector-database
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: aiohttp>=3.9.0
Requires-Dist: jieba>=0.42.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: onnxruntime==1.19.2
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dateutil>=2.8.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: sqlite-vec>=0.1.9
Requires-Dist: tokenizers==0.19.1
Provides-Extra: dev
Requires-Dist: autoflake>=2.0.0; extra == 'dev'
Requires-Dist: flake8>=7.0.0; extra == 'dev'
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Provides-Extra: faiss
Requires-Dist: faiss-cpu>=1.14.0; extra == 'faiss'
Provides-Extra: full
Requires-Dist: faiss-cpu>=1.14.0; extra == 'full'
Requires-Dist: matplotlib>=3.0.0; extra == 'full'
Requires-Dist: networkx>=3.0; extra == 'full'
Requires-Dist: pyvis>=0.3.0; extra == 'full'
Provides-Extra: rerank
Requires-Dist: sentence-transformers>=3.0.0; extra == 'rerank'
Provides-Extra: viz
Requires-Dist: matplotlib>=3.0.0; extra == 'viz'
Requires-Dist: networkx>=3.0; extra == 'viz'
Requires-Dist: pyvis>=0.3.0; extra == 'viz'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License">
  <img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="Python 3.10+">
  <img src="https://img.shields.io/badge/MCP-2025--03--26-purple" alt="MCP">
  <img src="https://img.shields.io/badge/version-0.1.70-green" alt="Version 0.1.69">
  <img src="https://img.shields.io/github/stars/j19800/MemALL?style=social" alt="GitHub stars">
</p>

<h1 align="center">MemALL</h1>
<p align="center"><strong>Multi-agent Memory OS</strong> — 本地优先的 AI Agent 持久化记忆系统</p>

<p align="center">
  <i>7 MCP tools · 3 priority + 11-layer memory lifecycle · 31-step self-evolving pipeline · Knowledge graph · Multi-agent shared memory · <b>Hook-driven automation</b></i>
</p>

<p align="center">
  <b>English</b> · <a href="README.zh-CN.md">中文</a> · <a href="CHANGELOG.md">Changelog</a>
</p>

---

## ✨ What is MemALL?

MemALL gives AI Agents **memory that persists across sessions, tools, and agents**. Install it, connect it to Claude Desktop / Cursor / Cline / any MCP client, and your agents stop starting from scratch every conversation.

```bash
pip install memall-os          # install (lightweight, no AI models)
pip install memall-os[full]    # install + viz + FAISS
pip install memall-os[rerank]  # install + cross-encoder reranking (HEAVY: ~1.8GB, requires PyTorch)
memall init                    # initialize
memall start                   # start services
memall hook list               # list registered lifecycle hooks
memall hook register post_capture --action log  # register a hook
# → MCP ready at http://127.0.0.1:9876/mcp
# → MCP ready at stdio (add to your mcp.json)
```

Then in any MCP client:

```json
{
  "mcpServers": {
    "memall": {
      "command": "memall",
      "args": ["serve"]
    }
  }
}
```

```python
# /capture "Decided to go open-source: MIT license, GitHub-first distribution"
# /retrieve "pricing decision"
# → finds the memory above, plus its context, timestamp, and relations
```

---

## 🔥 Why MemALL?

### 🧠 14-Level Memory Lifecycle (3 Priority + 11 Lifecycle Layers)

Not just "store and retrieve". Every memory has a semantic level with exclusion-based priority classification, epoch detection, and echo scoring:

| Level | Meaning | Example |
|-------|---------|---------|
| **P0/P1/P2** | **Planning** (priority layers) | "Ship v0.1.0 by June 23" |
| L1 | Raw fact | "User reported login bug" |
| L2 | Convention | "Use exclusion chain for classifier" |
| L3 | Business idea | "MemPort — cross-platform memory migration" |
| L4 | Decision | "Use FastAPI, not Flask" |
| L5 | Discussion | Multi-agent debate with convergence |
| **L6** | **Self-reflection** | "What went well/badly this session" |
| **L7** | **Preference** | "Prefers SMALL response, English-only" |
| **L8** | **Edge-promoted** | Auto-promoted via knowledge graph connectivity |
| **L9** | **Distillation** | Same-agent, same-category knowledge compression ("10k conversations → 200 knowledge nodes") |
| **L10** | **System insight** | **Cross-category** insight — same agent spanning ≥2 distinct categories |
| **L11** | **Domain intelligence** | **Cross-project / cross-agent** domain patterns |

> **Scope note:** "Cross-domain" at L10 means *cross-category within one agent* (produced by `integrate_step`). Truly *cross-agent* insight is produced separately by the `cross_agent` pipeline step, which can also emit L10 (cross-agent L9 aggregation) and L11 (domain patterns recurring across ≥5 agents). The two mechanisms are distinct.

**30 core steps + 5 optional** auto pipeline (see `list_pipeline_steps()`): enriches → classifies → detects epochs → reflects → distills → integrates → observes. No manual CRUD needed.

### 🔗 Knowledge Graph

Memories are connected by typed edges (`refines`, `cites`, `contradicts`, `supersedes`, `extends`). Traverse from "bug #123" to "fix PR #456" to "retrospective L6 reflection" in one hop.

```
/capture "Bug #123: OOM on large datasets"
/connect 123 456 --relation "resolved_by"
/traverse 123
→ finds #456 (fix), #789 (regression test), #912 (L6 reflection)
```

### 🤝 Multi-Agent Shared Memory

Claude, opencode, Codex, WorkBuddy — all reading from and writing to the same memory base. Cross-agent queries, fact extraction, and active push delivery are single MCP calls.

```
/fed_query "architecture decision last week" agent_name="claude"
→ returns cross-agent results with source attribution
```

### 🔄 Self-Evolving Pipeline

MemALL doesn't just store — it **improves itself**:

1. **L6 Reflection** — auto-reviews work quality, identifies patterns, corrects mistakes
2. **L7 Preference Extraction** — learns user preferences from interaction patterns
3. **Hook-Driven Automation** — post-capture auto-triggers lightweight pipeline + discussion convergence; pre-retrieve injects pending work; post-pipeline reports rich summaries
4. **L9 Distillation** — compresses raw conversations into structured knowledge
5. **L11 Domain Intelligence** — cross-project pattern detection
6. **Forget & Adaptive** — TTL-based decay, low-value cleanup, automatic re-indexing
7. **OODA Loop** — Observe → Orient → Decide → Act, no human intervention

### 🏠 100% Local

SQLite + FTS5 + vector search. Zero cloud dependency. Your data stays on your machine.

---

## 🛠️ MCP Tool Surface (7 server endpoints)

> **Count clarification:** The gateway exposes **7 top-level MCP tools** (`memall_write`, `memall_read`, `memall_persona`, `memall_discussion`, `memall_federation`, `memall_system`, `memall_hooks_recent`). Each is a *consolidated* tool with many `action`s — the table below lists the individual actions, which may read like "30+ tools" but are dispatched through those 7 endpoints. This is why older docs said "6 consolidated tools": the count depends on whether `memall_hooks_recent` is counted. Source of truth: `src/memall/mcp/tools/__init__.py`.

| Category | Tools |
|----------|-------|
| **Memory CRUD** | `capture`, `retrieve`, `update`, `smart_store`, `store_batch` |
| **Knowledge Graph** | `connect`, `traverse`, `timeline` |
| **Search** | `vector_search`, FTS5 full-text search, `hybrid_search` (FTS5+vec0 RRF, optional cross-encoder[¹]), `memall_search` |
| **Session** | `session_start`, `session_end`, `session_summary` |
| **Identity & Persona** | `persona`, `persona_profile`, `identity`, `ask`, `memall_persona action=identity` |
| **Discussion & Decision** | `discussion_create`, `discussion_respond`, `discussion_status`, `trace` |
| **Distillation** | `memall_system action=distill` |
| **Federation** | `fed_query`, `fed_publish`, `fed_deliver`, `fed_conflicts`, `fed_inject`, `fed_extract` |
| **Hub Sync** | `hub_connect`, `hub_sync` |
| **Pipeline & Evolution** | `run_pipeline`, `reflect_interact`, `forget`, `adaptive`, `index_rebuild`, `memall_write action=forget`, `memall_system action=adaptive` |
| **Hook Activity** | `memall_hooks_recent` — view async pipeline runs, notifications, and reminder checks |
| **Security & Ops** | `security`, `ops`, `gateway`, `db`, `memall_system action=db`, `memall_system action=security`, `memall_write action=ops` |
| **Onboarding** | `onboarding` |
| **Export/Import** | `export`, `import`, `sync` |

---

## 🚀 Quick Start

```bash
# One-shot install + start (recommended)
pip install memall-os && memall quickstart

# Or from source
git clone https://github.com/j19800/MemALL.git
cd MemALL
python install.py
```

`memall quickstart` initialises the database, starts the gateway, wires up the
MCP config, and opens the browser — nothing else to configure.

Then start recording:

```bash
memall capture "Project X: decided to use FastAPI, reason: async support"
memall capture "Fixed OOM bug in NLP pipeline — root cause: vector dimension mismatch"
memall retrieve "FastAPI decision"
```

---

## 📊 Why Not Competitors?

| Feature | MemALL | Mem0 | Letta | Zep |
|---------|--------|------|-------|-----|
| **Memory model** | 14 levels (P0/P1/P2 + L1–L11) | user/session | agent/memory-block | session/summary |
| **Knowledge graph** | ✅ Native + traversal | ❌ | ❌ | ❌ |
| **Self-evolving pipeline** | ✅ 31-step auto + 5 optional | ❌ | ❌ | ❌ |
| **Multi-agent shared** | ✅ Federation + active push | ❌ | Same agent only | ❌ |
| **Decision tracking** | ✅ Arc lifecycle | ❌ | ❌ | ❌ |
| **Discussion convergence** | ✅ Multi-agent auto | ❌ | ❌ | ❌ |
| **Protocol** | **MCP (native)** | REST API | REST + gRPC | REST API |
| **Local-first** | ✅ SQLite | ❌ Cloud | PostgreSQL | ❌ Cloud |
| **Open source** | ✅ MIT | ⚠️ Partial | ✅ | ⚠️ Partial |

---

## 📁 Project Structure

```
src/memall/
├── cli/          # CLI (40+ commands)
├── core/         # SQLite / NLP / vector search / event processor / echo scoring
├── api/          # FastAPI REST (35 routes)
├── mcp/          # MCP adapter (7 consolidated tool endpoints)
├── pipeline/     # 31-step auto pipeline + 5 optional (see list_pipeline_steps())
│   ├── observe/  # OODA observation step
│   ├── distill/  # L9/L10/L11 distillation, L7 preference, epoch detection
│   ├── classify/ # Exclusion-based priority classification
│   └── forget/   # TTL decay, L5 archive, low-value decay
├── graph/        # Knowledge graph + arc lifecycle management
├── federation/   # Cross-device memory sync + active push delivery
├── plugins/      # Hook/event-driven dashboard, exporter, notifier, scheduler
└── migrations/   # DB migrations
```

---

## 📝 Roadmap

- [x] **v0.1.0** — Core MCP server, 7 consolidated tool endpoints, CLI, PyPI package
- [x] **v0.1.69** — Repo made clonable again (`task_identity` + migrations 038/040 were live but uncommitted), goal `triage` gate now emits an action, skills layer transactions fixed for concurrency, policy refusals return 403 instead of 500
- [x] **v0.1.70** — Every remaining cross-agent read path closed: 
esources/read, ed_publish laundering/attribution forgery, and 	raverse/	imeline/	race now apply the trust filter; amily.db follows MEMALL_DB_PATH. Five red/blue duels now guard these.
- [ ] **v0.2.0** — Web dashboard, user system, Pro tier gating
- [ ] **v0.3.0** — Cloud sync, team collaboration, API gateway

### Known gaps (stated, not hidden)

- **`claim_task` is not atomic across processes.** `mcp/tasks.py` does read → modify → `UPDATE ... WHERE id = ?` with no state predicate and no row lock, so two concurrent claims end last-writer-wins and `claimed_by` is silently overwritten. Planned: add a state predicate or a unique index on `(task_id, status)`.
- **Goal-level block reasons are not observable.** Cards already carry a two-value `last_block_reason` enum; goals still `return None` when they are stuck, so "why this goal is not moving" is invisible.
- **The guard suite is slow by design.** Every guard replays all migrations via `init_db()`, so a full run is measured in hours rather than minutes. Treat a green card guard set as evidence for that set, not for the whole repository.
- **`check_fixes.py` leaves scratch directories** under `%TEMP%` (`mkdtemp` without `rmtree`).
- **Several P1 security items from the 2026-10-31 red/blue audit are still open.** `security permit` has no authorisation gate, `core.update()` does not enforce pinned fields, the gateway's loopback carve-out exempts too many endpoints from auth, and two competing identity-normalisation helpers (`normalize_agent_name` vs `canonical_agent`) still coexist. Read paths and attribution are closed; these are not.
- **Write-side attribution is not refused.** By the chosen policy ("writes pass, reads are locked") an explicit `agent_name` can still create work on another agent's behalf; impersonation is made observable via `wake_audit`/`ops_log` rather than rejected.

---

## 🤝 Contributing

PRs welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

- Report bugs → [Issues](https://github.com/j19800/MemALL/issues)
- Ask questions → [Discussions](https://github.com/j19800/MemALL/discussions)
- Browse the code → [Repository](https://github.com/j19800/MemALL)

---

## 📄 License

MIT License. See [LICENSE](LICENSE) for details.

> [¹] Cross-encoder reranking requires `pip install memall-os[rerank]` (downloads PyTorch + ~560MB model on first use). Core search works without it using RRF fusion of FTS5 + vec0.
