Metadata-Version: 2.4
Name: scm-memory
Version: 0.7.4
Summary: SCM: memory that works like yours — wake + sleep phases for AI agents.
Author-email: Saish Shinde <saish.shinde15@gmail.com>
Maintainer-email: Saish Shinde <saish.shinde15@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Saish15/sleepai
Project-URL: Documentation, https://github.com/Saish15/sleepai/blob/main/docs/DEPLOYMENT.md
Project-URL: Paper, https://github.com/Saish15/sleepai/blob/main/research/SCM_Final_Paper.pdf
Project-URL: Changelog, https://github.com/Saish15/sleepai/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/Saish15/sleepai/issues
Keywords: llm,memory,agent,lifelong-learning,consolidation,sleep,mcp,rag,lifelong-agent
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn>=0.23.0
Requires-Dist: pydantic>=2.0
Requires-Dist: networkx>=3.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: requests>=2.28.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: rich>=13.0
Requires-Dist: ollama>=0.1.0
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=2.2.0; extra == "embeddings"
Provides-Extra: llm
Requires-Dist: openai>=1.0.0; extra == "llm"
Provides-Extra: all
Requires-Dist: sentence-transformers>=2.2.0; extra == "all"
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: ollama>=0.1.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
Requires-Dist: jsonschema>=4.0; extra == "dev"
Dynamic: license-file

# SCM — Memory that works like yours

> **Mem0 stores facts your agent saw. SCM learns from them while it's idle.**

SCM (Sleep-Consolidated Memory) is the first open-source agent memory layer with both a **wake phase** and a **sleep phase** — like the only memory system in nature that actually works.

[📄 Paper (35 pages)](research/SCM_Final_Paper.pdf) · [🛠 Deployment guide](docs/DEPLOYMENT.md) · [🔌 Integrations](docs/INTEGRATIONS.md) · [🗺 Roadmap](docs/ROADMAP.md) · [📊 Benchmarks](docs/BENCHMARKS.md)

---

## What's different

Every other agent memory product (Mem0, MemGPT, A-Mem, RAG layers) does the same thing in different shapes: **store facts, retrieve facts**. They never think between sessions. Your agent forgets to think the moment you stop talking to it.

That's not how memory works in any system that does it well — including yours. **Sleep is when memory consolidates.** When you sleep, your hippocampus replays the day's experiences and your cortex abstracts patterns from them. You wake up with a *better* version of yesterday's understanding, not the same one.

SCM does both phases:

| Phase | What happens | Bio analog | SCM modules |
|---|---|---|---|
| **Wake** | Selective attention. Encoding-by-importance. Cue-driven retrieval. Contradiction handling. Bounded working memory (~7 items). | Hippocampal encoding, working memory, cue-driven recall | Phases 1-5 |
| **Sleep** | Pattern abstraction. Contradiction resolution. Adaptive forgetting. Knowledge-gap detection and curiosity-driven filling. Wake summary report. | NREM consolidation, REM dreaming, synaptic homeostasis | Phase 7 (M1-M6) |

**The result the user sees:** when they come back from being away, the agent reports what it noticed.

```
> What did you notice while I was away?

While you were away I noticed three things:
  • You've changed jobs — I've moved you from Northstar Robotics to Atlas Labs.
  • Your Tuesday-morning runs and Friday-night dinners with Mara have become weekly patterns.
  • You've mentioned 'OAuth flow' five times without explaining it; I read up on it.
    (Authorization protocol, redirect-based, token + scope.)
```

This moment is the product. **No other open-source memory system has it.**

---

## Five-line quickstart

```python
from src.chat.engine import ChatEngine

engine = ChatEngine(profile="chatbot")
engine.chat("Hi, I'm Saish. I run every Tuesday morning.")
engine.chat("Tuesday again — out for a 5K.")
engine.force_sleep("deep")
print(engine.wake_summary().narrative)
# → "While you were away I noticed Tuesday-morning running has become a pattern..."
```

Or, drop the SCM MCP server into Claude Desktop / Cursor / any MCP client and add five tools (`add_memory`, `search_memory`, `consolidate`, `wake_summary`, `forget`) automatically. See [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md).

---

## Install

```bash
git clone https://github.com/Saish15/sleepai.git
cd sleepai
python -m venv venv && source venv/bin/activate
pip install -e .
```

For the recommended privacy-first profile (free, local, no cloud calls):

```bash
ollama pull nomic-embed-text     # 274 MB embedding model (recommended)
ollama pull llama3.2:latest      # ~4 GB chat / extraction model
```

That's it. SCM auto-detects Ollama and uses it. Four deployment profiles documented in [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md):

| Profile | LLM | Embedding | Cost / 30 turns | Privacy |
|---|---|---|---|---|
| **A** Offline-only | heuristic regex | sentence-transformers MiniLM | $0 | 100% local |
| **B** Ollama-only (recommended) | Ollama llama3 | Ollama nomic-embed-text | $0 | 100% local |
| **C** Hybrid | DeepSeek-chat | Ollama nomic-embed-text | ~$0.04 | text→cloud, vectors local |
| **D** All-cloud | OpenAI gpt-4o-mini | OpenAI text-embedding-3-large | ~$0.06 | all→cloud |

---

## Works seamlessly with any LLM and any harness

SCM doesn't care which LLM you use. Concept extraction goes through `LLMExtractor`; switch providers with one env var:

```bash
LLM_PROVIDER=ollama       # local, free
LLM_PROVIDER=deepseek     # cheap cloud
LLM_PROVIDER=openai       # premium
# Anthropic / Voyage / Together / any OpenAI-compatible endpoint also works
```

Same for embeddings — sentence-transformers, Ollama, or any OpenAI-compatible provider.

Drop SCM behind any agent framework:
- **MCP server** (Claude Desktop, Cursor, ChatGPT-with-MCP) — `scm mcp` in your config
- **REST API** (`/v1/memories`, `/v1/wake-summary`, etc.) — Mem0-shaped, OpenAPI 3.1 spec at `/v1/openapi.json`
- **Python SDK** — `from src.integrations.langchain_adapter import SCMClient`
- **JavaScript SDK** — `import { SCM } from "scm-memory"` (Node 18+, Bun, browsers, Edge runtime)
- **LangChain memory adapter** — drop-in `BaseChatMemory` subclass
- **Plain HTTP** — POST `/v1/memories` from anything

Tool definitions exported in OpenAI / Anthropic / Gemini / OpenAPI formats from one source. See [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md) for 7 integration recipes.

---

## What SCM is NOT

- ❌ Not a vector database. It uses one (NetworkX in-memory + SQLite/Postgres backing), but the value is the lifecycle, not the index.
- ❌ Not an LLM. Bring your own.
- ❌ Not a chat UI. It's the memory backend; the UI is your problem (or wire it to the included `/static` demo page).
- ❌ Not a fact-extraction prompt. Mem0 is essentially a prompt + a vector DB. SCM is a complete memory pipeline.
- ❌ Not 100% production-polished yet. v0.7.x is research-grade with strong tests; the deployment guide closes most rough edges. See [`docs/STATUS.md`](docs/STATUS.md) for an honest current-state read.

---

## Honest comparison

We don't claim to dominate every memory benchmark. We dominate a different axis.

| Capability | Vector DB | MemGPT | Mem0 | A-Mem | **SCM** |
|---|---|---|---|---|---|
| Vector retrieval | ✅ | ✅ | ✅ | ✅ | ✅ |
| Working-memory bound | ❌ | ❌ | ❌ | ❌ | ✅ |
| Event-structured encoding | ❌ | ❌ | ❌ | ❌ | ✅ |
| Spreading-activation retrieval | ❌ | ❌ | ❌ | ❌ | ✅ |
| Contradiction-safe versioning | ❌ | ❌ | ❌ | ❌ | ✅ |
| Sleep-stage consolidation | ❌ | ❌ | ❌ | ❌ | ✅ |
| Schema extraction (REM) | ❌ | ❌ | ❌ | ❌ | ✅ |
| Wake-summary endpoint | ❌ | ❌ | ❌ | ❌ | ✅ |
| Curiosity-driven gap-filling | ❌ | ❌ | ❌ | ❌ | ✅ |
| Idle-aware autonomous learning | ❌ | ❌ | ❌ | ❌ | ✅ |
| **Cleaner fact-recall benchmark (LoCoMo)** | mid | mid | **best** | mid | weaker (we don't optimize for this) |

LoCoMo, the standard agent-memory benchmark, rewards Mem0's architecture and penalizes SCM's. We document this honestly in [the paper §9.5](research/SCM_Final_Paper.pdf) and built [LoCoMo++](research/SCM_Final_Paper.pdf) (workload-sensitivity benchmark) showing the win pattern is architectural, not absolute.

---

## Status

- **322 regression tests** passing (`pytest tests/ -q`)
- **143 focused regression tests** for Phase 7 + retrieval (`pytest tests/test_*spreading* tests/test_*idle* tests/test_*curiosity* -q`)
- **16/16 brutal LangChain harness scenarios** passing (multi-day persona, contradiction, idle wake-summary, multi-user isolation, failure mode)
- **5,561× p50 latency speedup** on `add_memory` since v0.7.2 (async ingest)
- **5-40× RAM saved at multi-user scale** since v0.7.3 (embedding-model singleton)

See [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) for every measured number with reproduction instructions.

---

## Architecture

```
                ┌───────────────────────────────┐
                │  YOUR AGENT / HARNESS         │
                │  (LangChain, Claude, custom)  │
                └─────────────┬─────────────────┘
                              │
                              ▼
        ┌───────────────────────────────────────────┐
        │  SCM CORE                                  │
        │  Wake-phase: encode → bind → retrieve      │
        │  Sleep-phase: consolidate → schema → gap   │
        │  Phase 7:  M1 idle daemon                  │
        │            M2 cross-session pool           │
        │            M3 schema extraction (REM)      │
        │            M4 wake-summary endpoint        │
        │            M5 curiosity engine             │
        │            M6 lifecycle policy             │
        └────────┬─────────────────────┬─────────────┘
                 │                     │
                 ▼                     ▼
       ┌───────────────────┐   ┌───────────────────┐
       │  LLM BACKEND      │   │  EMBEDDING        │
       │  Ollama/DeepSeek/ │   │  Ollama/OpenAI/   │
       │  OpenAI/etc.      │   │  sentence-trans.  │
       └───────────────────┘   └───────────────────┘
```

35-page paper at [`research/SCM_Final_Paper.pdf`](research/SCM_Final_Paper.pdf). Documents architecture, formal definitions for all 11 equations, brutal-testing methodology, ALB pilot, LoCoMo + LoCoMo++ honest comparisons, encoder-dependence analysis.

---

## Project status

This project is in **active development**, working toward a public launch. See [`docs/STATUS.md`](docs/STATUS.md) for an unvarnished read of where it stands as a product, and [`docs/ROADMAP.md`](docs/ROADMAP.md) for what's coming next. The paper is ready and staged for arXiv submission but **held until the product-readiness checklist completes** — papers without products fade.

---

## License

MIT. See [`LICENSE`](LICENSE).

---

## Contact

`blobopera@proton.me`
