Metadata-Version: 2.4
Name: opencode-semantic-memory
Version: 0.1.0
Summary: Persistent semantic memory system for OpenCode sessions
Author-email: Gregory Havenga <ghavenga@gitlab.com>
License-Expression: MIT
Keywords: ai,assistant,mcp,memory,opencode
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27.0
Requires-Dist: lancedb>=0.4.0
Requires-Dist: markdown-it-py>=3.0.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pylance>=0.20.0
Requires-Dist: sentence-transformers>=2.2.0
Requires-Dist: starlette>=0.36.0
Requires-Dist: tomli>=2.0.0
Requires-Dist: uvicorn>=0.27.0
Requires-Dist: watchdog>=3.0.0
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# opencode-memory

Persistent semantic memory system for [OpenCode](https://opencode.ai) AI assistant sessions.

## Overview

opencode-memory is an MCP (Model Context Protocol) server that provides long-term memory capabilities for OpenCode sessions. It automatically learns from your workflow, stores decisions and procedures, and provides relevant context when needed.

**Key benefits:**
- Remember decisions, blockers, procedures, and facts across sessions
- Semantic search finds relevant memories even with different wording
- Session coordination prevents conflicts when running multiple OpenCode instances
- Project-scoped directives for per-project instructions
- Zero manual effort - learns passively from your workflow

## Features

- **Hybrid retrieval**: Combines full-text search (FTS5) with vector similarity search
- **Instant storage**: Memories stored immediately, embeddings computed async
- **Session coordination**: Tracks active sessions, supports item claiming to prevent conflicts
- **Contextual directives**: Global + project-specific instructions loaded at boot
- **Entity tracking**: Links memories to MRs, issues, epics (!123, #456, &789)
- **GitLab enrichment**: Fetches metadata for entities from GitLab API
- **Memory age**: All outputs show age to identify potentially outdated information

## Installation

### Quick Install (Recommended)

Works on both Linux and macOS:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

This will:
- Clone the repository to `~/.local/share/opencode-memory-install`
- Create a virtual environment and install dependencies
- Download the embedding model (~90MB, one-time, runs locally)
- Set up a background service:
  - **Linux**: systemd user service
  - **macOS**: launchd LaunchAgent
- Configure OpenCode integration
- Create `~/.config/opencode/AGENTS.md` with memory bootstrap
- Optionally bootstrap core directives (requires confirmation due to LLM costs)

### Cost Information

**Default background processing is free** - The memory daemon uses:
- Local embedding model (sentence-transformers, no API calls)
- Pattern-based extraction (regex, no LLM)
- Session summaries (stores conversations as-is, no LLM processing)

**LLM-based knowledge extraction is OFF by default** - There is an optional
feature that uses `opencode run` to analyze old conversations and extract
procedures, decisions, and facts. This:
- Runs every 6 hours, processing up to 50 conversations per cycle
- Makes LLM API calls for each conversation (significant cost potential)
- Is **disabled by default** to avoid unexpected charges

To enable LLM extraction (if you want it), add to `~/.config/opencode-memory/config.toml`:
```toml
[ingestion]
llm_extraction = true
```

**Optional directive bootstrapping** - The setup script can create initial
directives using `opencode inline` (~4 LLM API calls). This prompts for
confirmation and defaults to 'no' when running non-interactively.

To skip directive bootstrapping entirely:
```bash
SKIP_BOOTSTRAP=1 curl -fsSL .../setup.sh | bash
```

### From Source (Manual)

```bash
# Clone the repository
git clone https://gitlab.com/ghavenga/opencode-memory.git
cd opencode-memory

# Create virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e ".[dev]"

# Download the embedding model (runs once, ~90MB)
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
```

### From PyPI

```bash
pip install opencode-semantic-memory
```

## Quick Start

### 1. Start the Memory Server

The recommended setup runs a shared HTTP server that all OpenCode sessions connect to:

```bash
# Start manually
source .venv/bin/activate
python -m opencode_memory.http_server
```

Or install as a background service (auto-starts on login):

**Linux (systemd):**
```bash
# Copy and customize the service file
cp opencode-memory.service ~/.config/systemd/user/

# Edit to match your paths
nano ~/.config/systemd/user/opencode-memory.service

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now opencode-memory

# Check status
systemctl --user status opencode-memory
```

**macOS (launchd):**
```bash
# Copy the plist file
cp com.opencode.memory.plist ~/Library/LaunchAgents/

# Edit to match your paths (replace $HOME with actual path)
nano ~/Library/LaunchAgents/com.opencode.memory.plist

# Create log directory
mkdir -p ~/.local/state/opencode-memory

# Load the service
launchctl load ~/Library/LaunchAgents/com.opencode.memory.plist

# Check status
launchctl list | grep opencode
```

### 2. Configure OpenCode

Add to `~/.config/opencode/opencode.json`:

```json
{
  "mcp": {
    "memory": {
      "type": "remote",
      "url": "http://localhost:9824/mcp"
    }
  }
}
```

### 3. Configure the Agent

Add to `~/.config/opencode/AGENTS.md`:

```markdown
# Agent Instructions

At session start, always call `memory_get_boot_context` to load prior context, active sessions, and blockers.

## Memory System

The memory system provides persistent context across sessions.

### Proactive Usage

Call `memory_remember` when:
- You make a decision about approach
- You hit a blocker
- You learn how to do something
- You discover an interesting fact

Call `memory_recall(query)` before:
- Using an unfamiliar API
- Making assumptions about how something works

Call `memory_get_context(entity_ref)` before working on any MR/issue/epic.

Categories: decision, blocker, procedure, fact, event, directive
```

## Configuration

Create `~/.config/opencode-memory/config.toml`:

```toml
[identity]
user = "your-gitlab-username"  # Optional, auto-detected from git
instance = "gitlab.com"

[boot]
identity = true
active_sessions = true
hot_items = true
unresolved_blockers = true
recent_decisions = false
max_hot_items = 5

[ingestion]
watch_paths = ["~/.local/share/opencode/opencode.db"]
db_poll_interval = 30
llm_extraction = false
working_directory = "/path/to/your/projects"

[storage]
path = "~/.local/share/opencode-memory"
```

## MCP Tools

> **Note:** Tools are registered with the `memory_` prefix when accessed through the "memory" MCP server.
> For example, `recall` becomes `memory_recall` in OpenCode sessions.

### Core Memory Tools

| Tool | Description |
|------|-------------|
| `memory_recall(query, limit?, project?, category?, compact?)` | Semantic search across memories |
| `memory_remember(content, category, what?, why?, learned?, entities?, project?)` | Store a memory |
| `memory_get_context(entity_ref)` | Get all memories for !MR, #issue, or &epic |
| `memory_get_boot_context()` | Load startup context (identity, blockers, directives) |
| `memory_get_linked_memories(memory_id, link_types?)` | Get memories linked to a specific memory |

### Session Coordination

| Tool | Description |
|------|-------------|
| `memory_session_start(session_id, working_on?)` | Register session |
| `memory_session_end(session_id, summary?)` | End session with summary |
| `memory_session_heartbeat(session_id)` | Keep session alive |
| `memory_get_active_sessions()` | List active sessions |
| `memory_claim_item(session_id, item_ref)` | Claim exclusive ownership |
| `memory_release_item(session_id, item_ref)` | Release claimed item |

### Memory Management

| Tool | Description |
|------|-------------|
| `memory_resolve_blocker(memory_id)` | Mark blocker as resolved |
| `memory_unresolve_blocker(memory_id)` | Reopen a blocker |
| `memory_archive_memory(memory_id, reason?)` | Archive outdated memory (soft delete) |
| `memory_delete_memory(memory_id, also_delete_vector?)` | Permanently delete a memory |
| `memory_edit_memory(memory_id, content?, what?, why?, learned?)` | Edit memory content or metadata |
| `memory_bulk_archive(memory_ids?, category?, older_than_days?, reason)` | Archive multiple memories |
| `memory_consolidate_memory(days_stale?, project?)` | Find stale/duplicate memories |
| `memory_search_history(query, category?, limit?)` | Search with category filter |

### Backup & Transfer

| Tool | Description |
|------|-------------|
| `memory_export_memories(output_path?, project?, categories?, since_days?)` | Export to JSON |
| `memory_import_memories(input_path, dry_run?, skip_duplicates?)` | Import from JSON |

### Utilities

| Tool | Description |
|------|-------------|
| `memory_ingest_file(file_path)` | Manually ingest a markdown file |
| `memory_enrich_entity(entity_ref, project?)` | Fetch GitLab metadata |
| `memory_bootstrap_memory(path?)` | Scan project files for initial facts |
| `memory_log_session(summary, learnings?, entities?)` | Log session summary |
| `memory_memory_status()` | Check system health and queue status |

## Memory Categories

| Category | Use For |
|----------|---------|
| `decision` | Architectural choices, design decisions |
| `blocker` | Obstacles preventing progress |
| `procedure` | How-to knowledge, workflows |
| `fact` | Project-specific information |
| `event` | Significant occurrences |
| `directive` | Always-on instructions (global or project-scoped) |
| `plan` | Long-term goals and strategies |
| `idea` | Future possibilities, deferred considerations |
| `conversation` | Full conversation content (auto-generated) |

## Storage

All data is stored locally:

- **SQLite**: `~/.local/share/opencode-memory/memory.db` - Memories, entities, sessions, FTS index
- **LanceDB**: `~/.local/share/opencode-memory/vectors/` - Vector embeddings for semantic search

The daemon automatically:
- Cleans up old resolved blockers (>90 days)
- Archives old conversations (>180 days)
- Compacts LanceDB versions (keeps last 10)

## Environment Variables

| Variable | Description |
|----------|-------------|
| `GITLAB_TOKEN` | Enable GitLab entity enrichment |
| `HF_HUB_OFFLINE=1` | Prevent model downloads (use cached) |
| `TRANSFORMERS_OFFLINE=1` | Prevent model downloads |
| `OPENCODE_MEMORY_HOST` | HTTP server bind address (default: 127.0.0.1) |
| `OPENCODE_MEMORY_PORT` | HTTP server port (default: 9824) |
| `OPENCODE_MEMORY_API_KEY` | Optional API key for authentication |
| `OPENCODE_MEMORY_RATE_LIMIT` | Requests per minute per client (default: 60) |

## Development

```bash
# Activate environment
source .venv/bin/activate

# Run tests
python -m pytest tests/ -v

# Lint
ruff check src/

# Type check
mypy src/

# Check memory stats
python -m opencode_memory.cli stats
```

## Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                      opencode-memory                            │
├─────────────────────────────────────────────────────────────────┤
│  MCP Server (HTTP/stdio)                                        │
│  └── 25+ tools for memory management                            │
├─────────────────────────────────────────────────────────────────┤
│  Background Daemon                                              │
│  ├── OpenCode DB Observer (polls for new sessions)              │
│  ├── Entity Enrichment (GitLab API)                             │
│  └── Cleanup (archives old memories, compacts vectors)          │
├─────────────────────────────────────────────────────────────────┤
│  Storage Layer                                                  │
│  ├── SQLite + FTS5 (memories, entities, sessions)               │
│  └── LanceDB (vector embeddings)                                │
├─────────────────────────────────────────────────────────────────┤
│  Embedding Engine                                               │
│  └── all-MiniLM-L6-v2 (384-dim, runs in subprocess)             │
└─────────────────────────────────────────────────────────────────┘
```

## Operations

### Log Rotation

When running as a systemd service, logs are managed by journald. To configure log rotation:

```bash
# Check current log size
journalctl --user -u opencode-memory --disk-usage

# Set max journal size (add to ~/.config/systemd/user.conf or override)
# Or create ~/.config/systemd/journald.conf.d/opencode-memory.conf:
cat > ~/.config/systemd/journald.conf.d/opencode-memory.conf << 'EOF'
[Journal]
SystemMaxUse=100M
MaxRetentionSec=7d
EOF

# Or manually vacuum old logs
journalctl --user --vacuum-time=7d
journalctl --user --vacuum-size=100M
```

For file-based logging, configure rotation in the systemd service:

```ini
[Service]
StandardOutput=append:/var/log/opencode-memory/server.log
StandardError=append:/var/log/opencode-memory/error.log
```

Then use logrotate (`/etc/logrotate.d/opencode-memory`):

```
/var/log/opencode-memory/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 $USER $USER
}
```

### Monitoring

The HTTP server exposes several endpoints:

| Endpoint | Description |
|----------|-------------|
| `/health` | Basic health check (returns 200 if healthy, 503 if degraded) |
| `/stats` | Detailed statistics (memories, cache, queue, links) |
| `/metrics` | Prometheus-format metrics |

Example monitoring with curl:
```bash
# Health check (for load balancers/monitoring)
curl -s http://localhost:9824/health | jq .

# Full stats
curl -s http://localhost:9824/stats | jq .

# Prometheus metrics
curl -s http://localhost:9824/metrics
```

### CLI Tools

```bash
# Show comprehensive statistics
python -m opencode_memory.cli stats

# Ingest markdown files
python -m opencode_memory.cli ingest /path/to/notes --recursive

# Archive old memories
python -m opencode_memory.cli cleanup --dry-run

# Enrich entities with GitLab metadata
python -m opencode_memory.cli enrich --limit 100
```

## Troubleshooting

### Check service status

**Linux:**
```bash
systemctl --user status opencode-memory
journalctl --user -u opencode-memory -f
```

**macOS:**
```bash
launchctl list | grep opencode
tail -f ~/.local/state/opencode-memory/server.log
tail -f ~/.local/state/opencode-memory/server.error.log
```

### Check memory system health
Use `memory_status` tool or:
```bash
python -c "
from opencode_memory.server import MemoryServer
import json
server = MemoryServer(enable_daemon=False)
print(json.dumps(server._get_status(), indent=2))
"
```

### Prometheus metrics
The HTTP server exposes metrics at `/metrics`:
```bash
curl http://localhost:9824/metrics
```

### Backup and restore
```bash
# Export all memories
memory_export_memories(output_path="/backup/memories.json")

# Import on new machine
memory_import_memories(input_path="/backup/memories.json", dry_run=True)  # Preview
memory_import_memories(input_path="/backup/memories.json")  # Actually import
```

### Reset memory database

**Linux:**
```bash
rm -rf ~/.local/share/opencode-memory/
systemctl --user restart opencode-memory
```

**macOS:**
```bash
rm -rf ~/.local/share/opencode-memory/
launchctl unload ~/Library/LaunchAgents/com.opencode.memory.plist
launchctl load ~/Library/LaunchAgents/com.opencode.memory.plist
```

## License

MIT
