Metadata-Version: 2.4
Name: opencode-semantic-memory
Version: 0.13.0
Summary: Persistent semantic memory system for OpenCode sessions
Author-email: Gregory Havenga <ghavenga@gitlab.com>
License-Expression: MIT
Keywords: ai,assistant,mcp,memory,opencode
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: croniter>=2.0.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: lancedb>=0.4.0
Requires-Dist: markdown-it-py>=3.0.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pylance>=0.20.0
Requires-Dist: sentence-transformers>=2.2.0
Requires-Dist: starlette>=0.36.0
Requires-Dist: tomli>=2.0.0
Requires-Dist: tree-sitter-javascript>=0.21.0
Requires-Dist: tree-sitter-python>=0.21.0
Requires-Dist: tree-sitter-ruby>=0.21.0
Requires-Dist: tree-sitter>=0.21.0
Requires-Dist: uvicorn>=0.27.0
Requires-Dist: watchdog>=3.0.0
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pandas>=2.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.1.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# opencode-memory

Persistent semantic memory system for [OpenCode](https://opencode.ai) AI assistant sessions.

## Overview

opencode-memory is an MCP (Model Context Protocol) server that provides long-term memory capabilities for OpenCode sessions. It automatically learns from your workflow, stores decisions and procedures, and provides relevant context when needed.

**Key benefits:**
- Remember decisions, blockers, procedures, and facts across sessions
- Semantic search finds relevant memories even with different wording
- Session coordination prevents conflicts when running multiple OpenCode instances
- Project-scoped directives for per-project instructions
- Zero manual effort - learns passively from your workflow

## Features

- **Hybrid retrieval**: Combines full-text search (FTS5) with vector similarity search
- **Instant storage**: Memories stored immediately, embeddings computed async
- **Session coordination**: Tracks active sessions, supports item claiming to prevent conflicts
- **Contextual directives**: Global + project-specific instructions loaded at boot
- **Entity tracking**: Links memories to MRs, issues, epics (!123, #456, &789)
- **GitLab enrichment**: Fetches metadata for entities from GitLab API
- **Memory age**: All outputs show age (`now`, `5m`, `2h`, `3d`, `2w`, `1mo`, `2y`) to identify potentially outdated information
- **Proactive context injection**: Automatically injects relevant memories before each LLM call (OpenCode only, via plugin)
- **Agent-agnostic LLM support**: Works with OpenCode, Claude CLI, GitLab Duo, Ollama, or custom commands
- **GitLab bulk ingest (optional)**: Index an entire GitLab project's issues and MRs into a separate database for weak semantic matching in `memory_recall`, with incremental daily sync to pick up new and updated issues/MRs
- **Queue defer/postpone**: Defer queue items you're waiting on and have them automatically resurface when a linked reminder fires
- **Secret-guard**: Reversible redaction of credentials from ingested content. Secrets are masked with placeholders in transit (so they never reach the model or memory) and restored only on a safe round-trip (writing back to the same file), eliminating false-positive corruption; at true exposure sinks (memory, comments, foreign files) they are never restored. Includes pattern auto-discovery, an exclude-paths allowlist, and a `forget` tool to undo a false positive. Set `reversible: false` (or `SECRET_GUARD_REVERSIBLE=0`) for the legacy one-way mode. See [docs/SECRET_GUARD.md](docs/SECRET_GUARD.md) and the [hybrid design](docs/SECRET_GUARD_HYBRID_DESIGN.md).
- **Tiered Docker sandboxing (optional)**: Run LLM-backed background tasks in a hardened container with per-task least-privilege GitLab tokens and a scrubbed LLM credential (default off; `mode = "host"`)
- **Migration system**: Versioned schema migrations with structured blockers/remediation and data-safety checks
- **Effectiveness judging**: Optional LLM material-influence judging of recalled memories to track which memories actually help
- **Optional Rust runtime**: Drop-in Rust backend sharing the same stores and HTTP/MCP API; selectable at install time
- **Version visibility**: `/health` and `/stats` report the running server version so you can confirm an upgrade took effect

## Prerequisites

- **Python 3.11+** (verify with `python3 --version` - must show 3.11 or higher)
- **git** (for cloning the repository)
- **venv** module (usually included with Python, or install via `python3-venv` package)

On Ubuntu/Debian:
```bash
sudo apt install python3.11 python3.11-venv git
```

On macOS (with Homebrew):
```bash
brew install python@3.11 git
```

## Installation

### From PyPI

```bash
pip install opencode-semantic-memory
```

### Quick Install Script

Works on both Linux and macOS, sets up the daemon as a background service:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

This will:
- Clone the repository to `~/.local/share/opencode-memory-install`
- Create a virtual environment and install dependencies
- Download the embedding model (~90MB, one-time, runs locally)
- Set up a background service:
  - **Linux**: systemd user service
  - **macOS**: launchd LaunchAgent
- Configure OpenCode integration
- Optionally bootstrap core directives (requires confirmation due to LLM costs)

**Re-running the installer is safe and non-interactive.** Your choices (runtime,
startup mode, LLM provider, permissions) are saved to
`~/.config/opencode-memory/config.toml` and reused on subsequent runs — re-runs
just update the install and fix anything missing, without re-asking. To change
your configuration, force the prompts with `--reconfigure`:

```bash
# from a local clone
bash scripts/setup.sh --reconfigure

# piped
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash -s -- --reconfigure
```

### From Source

```bash
git clone https://gitlab.com/ghavenga/opencode-memory.git
cd opencode-memory
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

# Download the embedding model (required, ~90MB, runs locally)
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
```

**Important:** After installing from source, you must manually set up the service (see Quick Start below). The service files in the repo assume the quick install path - you'll need to update the paths to match your clone location.

### Cost Information

**Default background processing is free** - The memory daemon uses:
- Local embedding model (sentence-transformers, no API calls)
- Pattern-based extraction (regex, no LLM)
- Session summaries (stores conversations as-is, no LLM processing)

**LLM-based knowledge extraction is OFF by default** - There is an optional
feature that uses `opencode run` to analyze old conversations and extract
procedures, decisions, and facts. This:
- Runs every 6 hours, processing up to 50 conversations per cycle
- Makes LLM API calls for each conversation (significant cost potential)
- Is **disabled by default** to avoid unexpected charges

To enable LLM extraction (if you want it), add to `~/.config/opencode-memory/config.toml`:
```toml
[ingestion]
llm_extraction = true
```

### LLM Provider Configuration

The memory system supports multiple LLM providers for knowledge extraction and session analysis. By default, it uses OpenCode, but you can configure alternatives:

```toml
# ~/.config/opencode-memory/config.toml

# Option 1: OpenCode (default) - uses your configured OpenCode LLM
[llm]
provider = "opencode"

# Option 2: Anthropic Claude CLI (requires claude-cli package)
[llm]
provider = "claude"
model = "claude-sonnet-4-20250514"  # optional

# Option 3: GitLab Duo CLI (requires glab with Duo enabled)
[llm]
provider = "duo"

# Option 4: Ollama (local models)
[llm]
provider = "ollama"
model = "qwen2.5:14b"  # or any model you have pulled

# Option 5: Custom command
[llm]
provider = "custom"
command = "my-llm-cli"
args = ["--prompt"]  # prompt is appended after these args
```

**Note:** The proactive context injection feature (automatic memory loading) currently only works with OpenCode via the plugin system. Other LLM providers can still use all memory tools manually.

## Updating

To update an existing installation, re-run the quick install script. It is idempotent and will pull the latest code, refresh dependencies, and restart the background services:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

The update is **non-destructive** for your data:

- Your memory database at `~/.local/share/opencode-memory/memory.db` is not touched
- Your vector store at `~/.local/share/opencode-memory/vectors/` is not touched
- Your existing `~/.config/opencode-memory/config.toml` is preserved (the script only writes a default if one does not already exist)

The script's `rm -rf` operations are scoped to the install directory (`~/.local/share/opencode-memory-install`) and only trigger when the git checkout or virtual environment is in a broken state. Any uncommitted local changes inside the install directory will be discarded (`git reset --hard` + `git clean -fd`) before the pull.

If you want to be extra cautious before a major update, snapshot the data directory first:

```bash
# Consistent SQLite backup (handles WAL files correctly)
TS=$(date +%Y%m%d-%H%M%S)
BACKUP_DIR="$HOME/.local/share/opencode-memory-backups/$TS"
mkdir -p "$BACKUP_DIR"
sqlite3 "$HOME/.local/share/opencode-memory/memory.db" ".backup '$BACKUP_DIR/memory.db'"
cp -R "$HOME/.local/share/opencode-memory/vectors" "$BACKUP_DIR/vectors"
cp "$HOME/.config/opencode-memory/config.toml" "$BACKUP_DIR/config.toml"
```

After the update, verify the service is healthy:

```bash
curl -s http://127.0.0.1:9824/health
# {"status":"ok","server":"opencode-memory","daemon":{"running":true}}
```

## Quick Start

### 1. Start the Memory Services

The recommended setup uses 3 background services (see [ARCHITECTURE.md](ARCHITECTURE.md)):

**Using the setup script (recommended):**
```bash
# The setup script handles everything including service installation
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

**Manual start (development/testing):**
```bash
source .venv/bin/activate

# Terminal 1: HTTP/MCP server
python -m opencode_memory.http_server

# Terminal 2: File watcher daemon
python -m opencode_memory.daemon

# Terminal 3: Background worker
python -m opencode_memory.jobs.worker
```

**Install services manually (Linux systemd):**
```bash
mkdir -p ~/.config/systemd/user
INSTALL_DIR="$PWD"  # Run from your clone directory

# Copy service files
for svc in opencode-memory opencode-memory-daemon opencode-memory-worker; do
    sed "s|%h|$HOME|g" "$INSTALL_DIR/${svc}.service" | \
    sed "s|\.local/share/opencode-memory-install|${INSTALL_DIR#$HOME/}|g" \
    > ~/.config/systemd/user/${svc}.service
done

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now opencode-memory opencode-memory-daemon opencode-memory-worker

# Check status
systemctl --user status opencode-memory*
```

**Install services manually (macOS launchd):**

See the setup script for plist templates, or run the setup script which handles this automatically.

### 2. Configure OpenCode

Add to `~/.config/opencode/opencode.json`:

```json
{
  "mcp": {
    "memory": {
      "type": "remote",
      "url": "http://localhost:9824/mcp"
    }
  }
}
```

### 3. Enable Proactive Context Injection (OpenCode Only)

The proactive context plugin automatically injects relevant memories before each LLM call, so the AI has historical context without needing explicit `memory_recall` calls.

**Requirements:**
- OpenCode (not supported with other LLM providers)
- Node.js 20+ (for building the plugin)

**Setup:**

The setup script automatically builds and configures the plugin if Node.js 20+ is available. To verify or configure manually:

```json
// ~/.config/opencode/opencode.json
{
  "mcp": { ... },
  "plugin": ["/path/to/opencode-memory-install/plugin"]
}
```

**How it works:**
1. User sends a message → Plugin extracts entities (!MR, #issue, @user)
2. Plugin calls memory server for relevant context
3. Context is injected into the system prompt before LLM call
4. AI sees relevant history automatically

**Thinking-aware injection:** The plugin also analyzes the assistant's responses to inject context mid-conversation. If the AI is "thinking" about a topic it has prior knowledge of, relevant memories are injected before the next LLM call.

**Relevance filtering:** Items with semantic similarity below 0.35 are filtered out to reduce noise. This threshold was tuned based on effectiveness analysis showing low-relevance items had 0% actual usefulness.

**Note:** If you're using a different LLM provider (Claude, Duo, Ollama), you'll need to call memory tools explicitly - proactive injection only works with the OpenCode plugin.

For more details, see the [plugin documentation](plugin/README.md).

### 4. Boot Context (Automatic)

When using the proactive context plugin (step 3), boot context is **automatically injected** at session start. This includes:
- Active parallel sessions (collision warning)
- Standing instructions (boot gates)
- Unresolved blockers
- Due reminders
- Memory usage instructions

**No manual configuration needed** - the plugin handles everything.

**Legacy AGENTS.md:** If you have an existing `~/.config/opencode/AGENTS.md` with `memory_get_boot_context()` instructions, you can remove it - boot context is now auto-injected. The setup script will offer to clean this up.

**Without the plugin:** If you're not using the proactive context plugin (e.g., using Claude CLI or Ollama), you'll need to call `memory_get_boot_context()` manually at session start

## Configuration

Create `~/.config/opencode-memory/config.toml`:

```toml
[identity]
user = "your-gitlab-username"  # Optional, auto-detected from git
instance = "gitlab.com"

[boot]
identity = true
active_sessions = true
hot_items = true
unresolved_blockers = true
recent_decisions = false
max_hot_items = 5

[ingestion]
watch_paths = ["~/.local/share/opencode/opencode.db"]
db_poll_interval = 30
llm_extraction = false
working_directory = "/path/to/your/projects"

[storage]
path = "~/.local/share/opencode-memory"
```

## MCP Tools

> **Note:** Tools are registered with the `memory_` prefix when accessed through the "memory" MCP server.
> For example, `recall` becomes `memory_recall` in OpenCode sessions.

### Core Memory Tools

| Tool | Description |
|------|-------------|
| `memory_recall(query, limit?, project?, category?, compact?)` | Semantic search across memories |
| `memory_remember(content, category, what?, why?, learned?, entities?, project?)` | Store a memory |
| `memory_get_context(entity_ref)` | Get all memories for !MR, #issue, or &epic |
| `memory_get_boot_context()` | Load startup context (identity, blockers, directives) |
| `memory_get_linked_memories(memory_id, link_types?)` | Get memories linked to a specific memory |

### Session Coordination

| Tool | Description |
|------|-------------|
| `memory_session_start(session_id, working_on?)` | Register session |
| `memory_session_end(session_id, summary?)` | End session with summary |
| `memory_session_heartbeat(session_id)` | Keep session alive |
| `memory_get_active_sessions()` | List active sessions |
| `memory_claim_item(session_id, item_ref)` | Claim exclusive ownership |
| `memory_release_item(session_id, item_ref)` | Release claimed item |

### Memory Management

| Tool | Description |
|------|-------------|
| `memory_resolve_blocker(memory_id)` | Mark blocker as resolved |
| `memory_unresolve_blocker(memory_id)` | Reopen a blocker |
| `memory_archive_memory(memory_id, reason?)` | Archive outdated memory (soft delete) |
| `memory_delete_memory(memory_id, also_delete_vector?)` | Permanently delete a memory |
| `memory_edit_memory(memory_id, content?, what?, why?, learned?)` | Edit memory content or metadata |
| `memory_bulk_archive(memory_ids?, category?, older_than_days?, reason)` | Archive multiple memories |
| `memory_consolidate_memory(days_stale?, project?)` | Find stale/duplicate memories |
| `memory_search_history(query, category?, limit?)` | Search with category filter |

### Backup & Transfer

| Tool | Description |
|------|-------------|
| `memory_export_memories(output_path?, project?, categories?, since_days?)` | Export to JSON |
| `memory_import_memories(input_path, dry_run?, skip_duplicates?)` | Import from JSON |

### Reminders

| Tool | Description |
|------|-------------|
| `memory_set_reminder(reminder_at, content?, memory_id?, recurrence?)` | Set time-triggered reminder |
| `memory_clear_reminder(memory_id)` | Dismiss a reminder |
| `memory_advance_reminder(memory_id)` | Move recurring reminder to next occurrence |
| `memory_mark_procedure_reviewed(memory_id)` | Mark procedure as still accurate |
| `memory_set_review_interval(memory_id, interval_days)` | Set procedure review frequency |

### Knowledge Graph

| Tool | Description |
|------|-------------|
| `memory_graph_aware(entity_ref, max_depth?)` | Check what knowledge exists for an entity |
| `memory_graph_explore(entity_ref, types?, limit?)` | Discover related entities |
| `memory_graph_retrieve(entity_refs, include_memories?)` | Get full content for entities |
| `memory_graph_status(project?)` | Knowledge graph statistics |

### Code Indexing

| Tool | Description |
|------|-------------|
| `memory_index_codebase(path?, incremental?)` | Index code structure (classes, functions) |
| `memory_initialize_workspace(path?, max_depth?)` | Scan directories for git repos |
| `memory_extract_concepts(batch_size?)` | Extract semantic concepts via LLM |
| `memory_search_code(query, symbol_types?)` | Search indexed code symbols |

### GitLab Spider

| Tool | Description |
|------|-------------|
| `memory_spider_search(entity_ref, project?, max_depth?)` | Crawl GitLab entity into knowledge graph |
| `memory_spider_status()` | Check spider queue status |
| `memory_spider_query(query?, author?, entity_type?)` | Search crawled GitLab content |

### Priority Queue

Work item queue for tracking todos, MRs to review, and authored MRs. Supports P0–P5 priorities with atomic claiming to prevent parallel session conflicts.

Items can also carry an optional `due_date`. As a deadline nears, an item's **effective** priority escalates toward P0 so milestone-driven work surfaces itself — without rewriting the stored base priority. Escalation only ever raises urgency, and pushing a due date out relaxes the item back to its base priority.

Escalation is **lead-time-proportional**: it tracks how far the item has travelled along its *runway* (from `created_at` to `due_date`), not an absolute number of days. The effective priority climbs incrementally from base toward P0 across the whole runway, clamping to P0 at/after the due date. So a task planned far in the future starts climbing early and gradually (a long horizon implies commensurate effort), while a short-runway task climbs quickly — both escalate over their own span. The curve is configurable (`start_fraction`, `gamma` shape, `max_runway_days` cap). Ordering uses effective priority, then soonest due date, then base priority, then age.

| Tool | Description |
|------|-------------|
| `memory_queue_get_next(project?, source?)` | Get next unclaimed item by effective priority |
| `memory_queue_claim(item_id, session_id)` | Claim item for your session |
| `memory_queue_release(item_id, session_id)` | Release claimed item back to pending |
| `memory_queue_complete(item_id, session_id, outcome_notes?)` | Mark item as completed |
| `memory_queue_skip(item_id, reason?, session_id?)` | Skip item (no longer relevant) |
| `memory_queue_list(status?, priority?, source?, project?)` | List queue items with filters (shows effective priority + due date) |
| `memory_queue_stats()` | Queue counts by status, priority, source, and due-date pressure |
| `memory_queue_reprioritize(item_id, new_priority)` | Change item base priority |
| `memory_queue_set_due_date(item_id, due_date?)` | Set or clear a due date (drives escalation) |
| `memory_queue_requeue(archived_id)` | Move archived item back to queue |
| `memory_queue_populate(items, refresh_id?)` | Bulk add/update queue items (accepts `due_date` per item) |

### Analytics

| Tool | Description |
|------|-------------|
| `memory_effectiveness_report(days?)` | Memory usage patterns, injection effectiveness, and suggestions |
| `memory_injection_metrics_trend(days?)` | Daily injection metrics over time for tracking improvements |
| `memory_background_report(days?)` | Background process activity |
| `memory_tool_metrics(action?, since_hours?)` | MCP tool call timing metrics |
| `memory_vector_index(action?)` | Manage vector search index |

### Utilities

| Tool | Description |
|------|-------------|
| `memory_ingest_file(file_path)` | Manually ingest a markdown file |
| `memory_enrich_entity(entity_ref, project?)` | Fetch GitLab metadata |
| `memory_bootstrap_memory(path?)` | Scan project files for initial facts |
| `memory_log_session(summary, learnings?, entities?)` | Log session summary |
| `memory_memory_status()` | Check system health and queue status |
| `memory_get_proactive_context(query, entities?, project?)` | Get context for proactive injection (plugin use) |

## GitLab Bulk Ingest (Optional)

Bulk ingest indexes **all issues and merge requests** from a GitLab project (default: `gitlab-org/gitlab`) into a **separate** database, so `memory_recall` can surface related upstream issues/MRs alongside your personal memories.

It is fully isolated from your main memory: a dedicated SQLite DB and vector store that can be disabled or deleted at any time without affecting your memories.

### How it works

1. **Fetch** - pulls issue/MR metadata (title, description, labels, state, dates) via the GitLab API, rate-limited and resumable.
2. **Keywords** - extracts the top ~50 keywords per entity (stop words, URLs, @mentions, and paths stripped). No LLM required.
3. **Index** - embeds title + description + labels with the same local `all-MiniLM-L6-v2` model used for memories.
4. **Search** - results merge into `memory_recall`, scored by vector similarity (weighted `0.8` vs personal memories) plus a small keyword-overlap boost (`0.1 * overlapping_keywords`).

All processing is local. The only network calls are to the GitLab API (fetching metadata) and a one-time embedding-model download.

### Data isolation

| | Main memory | Bulk ingest |
|---|---|---|
| SQLite | `~/.local/share/opencode-memory/memory.db` | `~/.local/share/opencode-memory/gitlab-org.db` |
| Vectors | `~/.local/share/opencode-memory/vectors/` | `~/.local/share/opencode-memory/gitlab-org.lance/` |

Bulk-ingested entities are never written into your main memory DB, knowledge graph, or consolidation/archiving flows. They are a read-time overlay only.

### Activation

```bash
# 1. Create the config file (~/.config/opencode-memory/bulk_ingest.toml)
opencode-memory bulk-ingest config --init

# 2. Fetch all issues + MRs (resumable; safe to re-run if interrupted)
#    For gitlab-org/gitlab this is ~580K entities and takes ~2 hours.
GITLAB_TOKEN=<your-token> opencode-memory bulk-ingest fetch

# 3. Build the vector index (embeds locally; ~3 hours for 580K on CPU)
opencode-memory bulk-ingest index --create-index

# 4. Check progress / counts at any time
opencode-memory bulk-ingest status
```

Once indexed, results appear automatically in `memory_recall` with `match_type: "bulk_ingest"` and a `bulk_ingest:<project>#<iid>` source.

### Keeping it up to date

To pick up newly created and recently updated issues/MRs, run an incremental sync. It fetches everything changed since the latest `updated_at` already in storage, so it catches both new entities and edits to existing ones:

```bash
# One-shot incremental sync
GITLAB_TOKEN=<your-token> opencode-memory bulk-ingest sync

# Explicit window
opencode-memory bulk-ingest sync --since 2026-06-01T00:00:00Z
```

For hands-off operation, the `BulkIngestDaemon` runs the incremental sync on a schedule (default every 24 hours, configurable via `sync_interval_hours`), optionally re-indexing new entities after each pass. Schedule it however you prefer (systemd timer, cron, or the daemon's own loop). The sync is poll-based rather than webhook-driven, so a daily cadence is the typical setup.

### CLI commands

| Command | Description |
|---------|-------------|
| `bulk-ingest config --init` | Create the default config file |
| `bulk-ingest fetch` | Fetch issue/MR metadata (resumable) |
| `bulk-ingest index --create-index` | Embed entities and build the IVF-PQ search index |
| `bulk-ingest index --reindex` | Re-embed everything (default skips already-indexed) |
| `bulk-ingest sync` | Incremental fetch of entities updated since last run |
| `bulk-ingest status` | Show entity counts, tiers, and DB size |
| `bulk-ingest enable` / `disable` | Toggle inclusion in `memory_recall` |

### Configuration

Key settings in `~/.config/opencode-memory/bulk_ingest.toml`:

```toml
[gitlab]
project = "gitlab-org/gitlab"   # any project path
api_rate_limit = 100            # requests per minute

[query]
enabled = true                  # include results in memory_recall
relevance_weight = 0.8          # discount vs personal memories

[summarizer]
enabled = false                 # LLM summaries are OPTIONAL and off by default
provider = "none"               # title + description is enough for search
```

> **Note:** LLM summarization is optional and disabled by default. Avoid the `opencode` summarizer provider - it has MCP access and would write entries into your main memory DB.

### Disabling

```bash
opencode-memory bulk-ingest disable        # stop including in recall (keeps data)
# or remove the data entirely:
rm -rf ~/.local/share/opencode-memory/gitlab-org.db ~/.local/share/opencode-memory/gitlab-org.lance
```

## Memory Categories

| Category | Use For |
|----------|---------|
| `boot_gate` | STOP trigger shown every session (≤400 chars, points to directive) |
| `directive` | Full standing instructions (loaded via recall) |
| `procedure` | How-to knowledge, workflows |
| `decision` | Architectural choices, design decisions |
| `blocker` | Obstacles preventing progress |
| `fact` | Project-specific information |
| `plan` | Sequence of steps with end state |
| `goal` | Sustained target/threshold (ongoing, no end state) |
| `idea` | Future possibilities, deferred considerations |
| `event` | Significant occurrences |
| `conversation` | Full conversation content (auto-generated) |

## Storage

All data is stored locally:

- **SQLite**: `~/.local/share/opencode-memory/memory.db` - Memories, entities, sessions, FTS index
- **LanceDB**: `~/.local/share/opencode-memory/vectors/` - Vector embeddings for semantic search

The daemon automatically:
- Cleans up old resolved blockers (>90 days)
- Archives old conversations (>180 days)
- Compacts LanceDB versions (keeps last 10)

## Environment Variables

| Variable | Description |
|----------|-------------|
| `GITLAB_TOKEN` | Enable GitLab entity enrichment |
| `HF_HUB_OFFLINE=1` | Prevent model downloads (use cached) |
| `TRANSFORMERS_OFFLINE=1` | Prevent model downloads |
| `OPENCODE_MEMORY_HOST` | HTTP server bind address (default: 127.0.0.1) |
| `OPENCODE_MEMORY_PORT` | HTTP server port (default: 9824) |
| `OPENCODE_MEMORY_API_KEY` | Optional API key for authentication |
| `OPENCODE_MEMORY_RATE_LIMIT` | Requests per minute per client (default: 60) |
| `SKIP_BOOTSTRAP` | Set to `1` to skip directive bootstrapping prompt in setup script |

## Development

```bash
# Activate environment
source .venv/bin/activate

# Run tests
python -m pytest tests/ -v

# Lint
ruff check src/

# Type check
mypy src/

# Check memory stats
python -m opencode_memory.cli stats
```

## Architecture

opencode-memory uses a **3-process architecture** to separate concerns:

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                           opencode-memory                                   │
├─────────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────────┐  ┌─────────────────────┐  ┌─────────────────────┐  │
│  │   HTTP/MCP Server   │  │       Daemon        │  │  Background Worker  │  │
│  │  (opencode-memory)  │  │ (opencode-memory-   │  │ (opencode-memory-   │  │
│  │                     │  │       daemon)       │  │       worker)       │  │
│  ├─────────────────────┤  ├─────────────────────┤  ├─────────────────────┤  │
│  │ • MCP tool handlers │  │ • File watching     │  │ • Embedding compute │  │
│  │ • Request/response  │  │ • OpenCode DB poll  │  │ • GitLab enrichment │  │
│  │ • Health endpoints  │  │ • Queue work items  │  │ • Memory linking    │  │
│  │                     │  │                     │  │ • Cleanup/archival  │  │
│  │                     │  │                     │  │ • LLM extraction    │  │
│  └─────────┬───────────┘  └─────────┬───────────┘  └─────────┬───────────┘  │
│            │                        │                        │              │
│            └────────────────────────┼────────────────────────┘              │
│                          ┌──────────▼──────────┐                            │
│                          │   Shared Storage    │                            │
│                          ├─────────────────────┤                            │
│                          │ • SQLite + FTS5     │                            │
│                          │ • LanceDB (vectors) │                            │
│                          │ • Work queues       │                            │
│                          └─────────────────────┘                            │
└─────────────────────────────────────────────────────────────────────────────┘
```

| Process | Purpose | Module |
|---------|---------|--------|
| **HTTP Server** | MCP tools, health endpoints | `opencode_memory.http_server` |
| **Daemon** | File watching, DB polling | `opencode_memory.daemon` |
| **Worker** | All heavy background processing | `opencode_memory.jobs.worker` |

This separation ensures:
- API requests are never blocked by background processing
- Worker can crash/restart without affecting the API
- Multiple workers can run in parallel for scaling

See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed documentation.

### Startup mode: always-on vs lazy

The HTTP server keeps the embedding model resident (~2 GB) so semantic search is
instant and shared across sessions. The installer lets you choose when the
services run:

| Mode | Boot behaviour | First-session cost | Idle RAM |
|------|----------------|--------------------|----------|
| **Always-on** (default) | Started at login | None | ~2 GB resident |
| **Lazy / on-demand** | Not started at boot | A few seconds (model load) | 0 until first use |

In **lazy mode** the services start automatically the first time an opencode
session connects (the `memory-mcp.sh` wrapper sees the marker at
`~/.config/opencode-memory/lazy` and runs `memory-ctl.sh start`), then stay
running until explicitly stopped. This suits low-RAM machines.

Choose lazy at install time interactively, or non-interactively:

```bash
OPENCODE_MEMORY_STARTUP=lazy bash scripts/setup.sh
```

Start/stop/inspect the stack at any time:

```bash
scripts/memory-ctl.sh status   # services + current mode
scripts/memory-ctl.sh start    # start all three (no-op if already healthy)
scripts/memory-ctl.sh stop     # stop all three, freeing the model's RAM
```

From inside a session you can free the RAM on demand with the **`memory_shutdown`**
MCP tool, which stops all three services. In lazy mode the next session restarts
them automatically; in always-on mode restart with `memory-ctl.sh start`.

## Operations

### Log Rotation

When running as a systemd service, logs are managed by journald. To configure log rotation:

```bash
# Check current log size
journalctl --user -u opencode-memory --disk-usage

# Set max journal size (add to ~/.config/systemd/user.conf or override)
# Or create ~/.config/systemd/journald.conf.d/opencode-memory.conf:
cat > ~/.config/systemd/journald.conf.d/opencode-memory.conf << 'EOF'
[Journal]
SystemMaxUse=100M
MaxRetentionSec=7d
EOF

# Or manually vacuum old logs
journalctl --user --vacuum-time=7d
journalctl --user --vacuum-size=100M
```

For file-based logging, configure rotation in the systemd service:

```ini
[Service]
StandardOutput=append:/var/log/opencode-memory/server.log
StandardError=append:/var/log/opencode-memory/error.log
```

Then use logrotate (`/etc/logrotate.d/opencode-memory`):

```
/var/log/opencode-memory/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 $USER $USER
}
```

### Monitoring

The HTTP server exposes several endpoints:

| Endpoint | Description |
|----------|-------------|
| `/health` | Basic health check (returns 200 if healthy, 503 if degraded) |
| `/stats` | Detailed statistics (memories, cache, queue, links) |
| `/metrics` | Prometheus-format metrics |

Example monitoring with curl:
```bash
# Health check (for load balancers/monitoring)
curl -s http://localhost:9824/health | jq .

# Full stats
curl -s http://localhost:9824/stats | jq .

# Prometheus metrics
curl -s http://localhost:9824/metrics
```

### CLI Tools

```bash
# Show comprehensive statistics
python -m opencode_memory.cli stats

# Ingest markdown files
python -m opencode_memory.cli ingest /path/to/notes --recursive

# Archive old memories
python -m opencode_memory.cli cleanup --dry-run

# Enrich entities with GitLab metadata
python -m opencode_memory.cli enrich --limit 100
```

## Troubleshooting

### Check service status

**Linux (systemd):**
```bash
# Check all services
systemctl --user status opencode-memory*

# View logs for specific service
journalctl --user -u opencode-memory -f         # Server logs
journalctl --user -u opencode-memory-daemon -f  # Daemon logs
journalctl --user -u opencode-memory-worker -f  # Worker logs

# Restart all services
systemctl --user restart opencode-memory opencode-memory-daemon opencode-memory-worker
```

**macOS (launchd):**
```bash
# Check all services
launchctl list | grep opencode

# View logs
tail -f ~/.local/state/opencode-memory/server.log
tail -f ~/.local/state/opencode-memory/daemon.log
tail -f ~/.local/state/opencode-memory/worker.log

# Restart all services
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl unload ~/Library/LaunchAgents/${svc}.plist 2>/dev/null
    launchctl load ~/Library/LaunchAgents/${svc}.plist
done
```

### Check memory system health
Use `memory_status` tool or:
```bash
# HTTP health endpoint
curl http://localhost:9824/health

# Detailed stats
curl http://localhost:9824/stats
```

### Prometheus metrics
The HTTP server exposes metrics at `/metrics`:
```bash
curl http://localhost:9824/metrics
```

### Backup and restore
```bash
# Export all memories
memory_export_memories(output_path="/backup/memories.json")

# Import on new machine
memory_import_memories(input_path="/backup/memories.json", dry_run=True)  # Preview
memory_import_memories(input_path="/backup/memories.json")  # Actually import
```

### Reset memory database

**Linux:**
```bash
systemctl --user stop opencode-memory opencode-memory-daemon opencode-memory-worker
rm -rf ~/.local/share/opencode-memory/
systemctl --user start opencode-memory opencode-memory-daemon opencode-memory-worker
```

**macOS:**
```bash
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl unload ~/Library/LaunchAgents/${svc}.plist 2>/dev/null
done
rm -rf ~/.local/share/opencode-memory/
for svc in com.opencode.memory com.opencode.memory.daemon com.opencode.memory.worker; do
    launchctl load ~/Library/LaunchAgents/${svc}.plist
done
```

### Upgrade from older versions

If you're upgrading from a version that used `opencode-memory-enrich.service`, the setup script will automatically migrate to the new 3-process architecture. Run the setup script again:

```bash
curl -fsSL https://gitlab.com/ghavenga/opencode-memory/-/raw/master/scripts/setup.sh | bash
```

This will:
1. Stop and disable the old `opencode-memory-enrich` service
2. Install the new `opencode-memory-daemon` and `opencode-memory-worker` services
3. Start all services

## License

MIT
