Metadata-Version: 2.4
Name: cli-memory-os
Version: 0.1.14
Summary: CLI-based Personal Knowledge Operating System
Author-email: Anirudh T <anirudh200584@gmail.com>
License-Expression: MIT
Keywords: RAG,Knowledge-Graph,Vector-Search,Personal-Assistant,LLM,Composio
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: composio==0.13.1
Requires-Dist: composio-langchain==0.13.1
Requires-Dist: langchain==1.3.9
Requires-Dist: langchain-groq==1.1.3
Requires-Dist: python-dotenv==1.2.2
Requires-Dist: qdrant-client==1.18.0
Requires-Dist: neo4j==6.2.0
Requires-Dist: sentence-transformers==5.6.0
Requires-Dist: textual>=0.80.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# Memory‑OS 🧠

[![PyPI version](https://img.shields.io/pypi/v/memory-os.svg)](https://pypi.org/project/memory-os/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Memory‑OS** is a local Personal Knowledge Operating System that syncs, indexes, and retrieves information across your GitHub repositories, emails, and Notion workspaces. It runs a unified interactive CLI, exposing hybrid keyword + semantic search and natural language QA powered by RAG, local embeddings, and a knowledge graph.

---

## 🏗️ Architecture

Memory-OS is built as a modular architecture consisting of ingestion, databases, scoring ranking engines, and a terminal user loop:

```mermaid
graph TD
    %% Ingestion
    subgraph Ingestion [1. Ingestion Layer]
        GH[GitHub Repos & Docs]
        GM[Gmail Inbox Messages]
        NT[Notion Page Contents]
        CP[Composio Integration Platform]
        GH --> CP
        GM --> CP
        NT --> CP
    end

    %% Storage & Indexing
    subgraph Storage [2. Storage & Indexing Layer]
        DB[(SQLite: workspace.db)]
        QD[(Qdrant: Qdrant Server)]
        N4J[(Neo4j Graph Database)]
        SQL_G[(SQLite Graph Fallback)]
        
        CP -->|Insert Metadata & Docs| DB
        DB -->|Text Chunks| CH[Chunking Core]
        CH -->|Local Embeddings| EM[SentenceTransformer]
        EM -->|Vectors| QD
        
        DB -->|Graph Construction| N4J
        DB -->|Graph Construction| SQL_G
    end

    %% Retrieval & RAG
    subgraph Retrieval [3. Retrieval & RAG Layer]
        HS[Hybrid Search Router]
        QD -->|Cosine Similarity| HS
        DB -->|Keyword Matching| HS
        N4J -->|Graph Lookups| HS
        SQL_G -->|Graph Fallback Lookups| HS
        
        HR[Hybrid Ranking Scoring]
        HS --> HR
        
        RAG[RAG Context Builder]
        HR -->|Merged Context| RAG
        
        LLM[Groq LLM Pipeline]
        RAG -->|Prompt Assembly| LLM
    end

    %% User Interaction
    subgraph User [4. Interface Layer]
        CLI[main.py: CLI Command Loop]
        CLI -->|Sync/Rebuild| Ingestion
        CLI -->|Search/Ask Queries| Retrieval
        LLM -->|Formatted Answer| CLI
    end
```

---

## 🗄️ Database Technology Rationale

Memory-OS adopts a **multi-model storage engine strategy**, selecting each technology to excel at its designated retrieve-and-rank role:

| Database | Selection Rationale |
| :--- | :--- |
| **SQLite (`workspace.db`)** | Chosen for lightweight structured storage. It holds raw documents, chunk segments, email metadata, and repository statistics, providing ACID compliance and ultra-fast exact keyword searches. |
| **Qdrant** | Chosen as a high-performance vector database optimized for storing and executing cosine similarity search queries on $384$-dimensional dense vector embeddings generated by Sentence-Transformers. |
| **Neo4j / Fallback SQLite** | Neo4j is utilized as a native graph database to map complex developer relationships (e.g. `Repository-[USES]->Technology` or `Email-[SENT_BY]->User`). If Neo4j is unreachable, it seamlessly falls back to a relational SQLite graph schema, preserving search functionality offline. |

---

## 🗄️ Workspace Structure

Memory-OS manages directories and configuration under the user's home folder (`~/.memory-os/`):
```
~/.memory-os/
├── config.toml                     # Global TOML settings configuration
├── active_profile                  # Stores name of the currently active profile
└── workspaces/
    ├── default/                    # Default workspace profile directory
    │   ├── workspace.db            # SQLite relational database
    │   ├── qdrant/                 # Qdrant local bind-mounted storage folder
    │   ├── neo4j/                  # Neo4j local bind-mounted storage folder
    │   ├── logs/                   # Log folder containing memory_os.log
    │   └── cache/                  # Chunker cache folder
    └── personal/                   # Personal workspace profile directory
```

---

## 📦 Setup & Installation

Ensure you have **Python >= 3.12** and **Docker + Docker Compose** installed.

### 1. Install via pip
You can install Memory-OS directly as a package:
```bash
pip install .
```
This registers the CLI entry point executable `memory-os` on your path.

### 2. Run the Initialization Wizard
Kick off the interactive wizard to verify system dependencies, configure API keys, spin up containers, and pre-warm model weights:
```bash
memory-os init
```

---

## 🚀 CLI Commands Reference

Memory-OS exposes a comprehensive CLI for administration:

### Core Daemon Lifecycle
- **`memory-os start`**: Spins up Neo4j and Qdrant database services in the background using Docker Compose.
- **`memory-os stop`**: Stops the database services while retaining data directories intact.

### Operations & Ingestion
- **`memory-os sync [--source SOURCE] [--rebuild]`**: Triggers incremental data imports from registered sources (GitHub, Gmail, Notion). Add `--rebuild` for full vector/graph resets.
- **`memory-os ask <question>`**: Runs natural language queries against the RAG retrieval pipeline.
- **`memory-os graph <repo>`**: Visualizes the relationships of an indexed repository in the terminal knowledge graph.

### Diagnostics & Monitoring
- **`memory-os doctor`**: Analyzes connection validation health across all endpoints and prints actionable troubleshooting tips on failure.
- **`memory-os monitor`**: Displays aggregated data latencies (indexing speed, search rates, LLM times) by parsing system log traces.
- **`memory-os benchmark`**: Performs query speed runs on keyword, semantic, hybrid searches, and RAG pipelines.
- **`memory-os logs [--tail N]`**: Tails the running logs of Memory-OS (rotates at 5MB, up to 3 backups).

### Configuration Management
- **`memory-os config show`**: Displays the active key-value configuration block.
- **`memory-os config get <key>`**: Fetches a nested key value (e.g. `groq.model`).
- **`memory-os config set <key> <value>`**: Sets a nested key value with type validation checks.
- **`memory-os config reset`**: Prompts and reverts all configurations to factory defaults.

### Workspace Profile Profiles
- **`memory-os workspace list`**: Lists all profiles (* marks active).
- **`memory-os workspace create <name>`**: Allocates a new workspace folder tree.
- **`memory-os workspace switch <name>`**: Switches the active profile context.
- **`memory-os workspace delete <name>`**: Wipes profile folder directories.
- **`memory-os workspace info`**: Displays detailed record metrics (nodes, vectors, sizes) for the active profile.

### Portability (Export / Import)
- **`memory-os export <backup-zip>`**: Compresses database schemas, configurations, and vector indices into a versioned zip package.
- **`memory-os import <backup-zip>`**: Overwrites the active workspace profile using files from an export package after validating versions and model compatibility.

---

## 🔌 Plugin System Architecture

Memory-OS features a structured connector registry. Every connector implements `BaseConnector` (`connectors/base.py`) and is registered using the `@register` decorator (`connectors/registry.py`):

```python
from connectors.base import BaseConnector
from connectors.registry import register

@register
class SlackConnector(BaseConnector):
    name = "Slack"
    slug = "slack"

    def authenticate(self) -> bool:
        # Check OAuth or API status
        return True

    def sync(self) -> dict:
        # Fetch channels and messages
        return {"synced": 42}

    def health(self) -> tuple[bool, str]:
        return True, "Connected"
```

To list registered plugins:
```bash
memory-os plugins
```

---

## 🛠️ Troubleshooting

- **Database Offline / Port conflicts**: If Neo4j (ports 7474/7687) or Qdrant (port 6333) fails to start, modify port configurations:
  ```bash
  memory-os config set neo4j.port_http 7475
  memory-os config set qdrant.port 6334
  memory-os start
  ```
- **Failing Diagnostics**: Run `memory-os doctor` to inspect status. It provides detailed actionable tips to address common environment issues.

---

## 📜 License
This project is licensed under the **MIT License**.
