# Differential KG Build Design — Issue #108

**Date**: 2026-03-09
**Issue**: https://github.com/bobmatnyc/mcp-vector-search/issues/108
**Title**: feat: differential KG build - only process changed files incrementally

---

## 1. Current Build Entry Point and Key Methods

### Entry Points (two paths, same logic)

**Primary path** — `mvs index kg` / `mvs kg build` / `mvs kg index`:

```
cli/commands/kg.py::_build_kg_impl()
  -> asyncio.run(_load_and_prepare_chunks())   # parent process: loads chunks from LanceDB, writes to temp JSON
  -> subprocess.run(_kg_subprocess.py ...)      # isolated subprocess: no LanceDB threads
       -> KnowledgeGraph(kg_path).initialize_sync()
       -> KGBuilder(kg, project_root).build_from_chunks_sync(chunks, ...)
```

**Legacy async path** (used by `build_from_database`):

```
kg_builder.py::KGBuilder.build_from_database(database, incremental=False)
  -> iterates database.iter_chunks_batched()
  -> filters out chunk_ids in _get_processed_chunk_ids() if incremental=True
  -> calls build_from_chunks(chunks, ...)
```

### Key Methods in `kg_builder.py`

| Method | Location | Purpose |
|--------|----------|---------|
| `build_from_chunks_sync()` | line 312 | Main synchronous build — separates code vs text chunks, calls `_extract_code_entity` for each, batch-inserts to Kuzu |
| `build_from_chunks()` | line 826 | Async version of above (same logic) |
| `build_from_database()` | line 4392 | Loads chunks from LanceDB, filters by processed IDs, delegates to `build_from_chunks()` |
| `_extract_code_entity()` | line 1135 | Extracts a `CodeEntity` + CALLS/IMPORTS/INHERITS/CONTAINS relationships from one chunk — pure Python, no DB writes |
| `_prescan_entity_names()` | (called at line 425) | Pre-populates `_entity_map` so forward-references resolve correctly |
| `_save_metadata()` | line 206 | Writes `kg_metadata.json` with chunk IDs, timestamps, counts |
| `_get_processed_chunk_ids()` | line 248 | Reads `kg_metadata.json`, returns set of previously processed chunk IDs |

### Chunk Loading in the Subprocess Path

The parent process (`_load_and_prepare_chunks` in `kg.py` lines 293-402):
1. Opens LanceDB via `ComponentFactory.create_standard_components()`
2. Calls `database.iter_chunks_batched(batch_size=5000)` — no file-hash filter available here
3. In incremental mode, reads `kg_metadata.json::processed_chunk_ids` and filters the iterator client-side
4. Serializes filtered chunks to a temp JSON file
5. Closes LanceDB before spawning the subprocess (critical — Kuzu segfaults with background LanceDB threads)

### Current "Already Processed" Tracking

`kg_metadata.json` (at `.mcp-vector-search/knowledge_graph/kg_metadata.json`) contains:
```json
{
  "last_build": "2026-03-01T12:00:00+00:00",
  "source_chunk_count": 77432,
  "source_chunk_id_hash": "a3f2b1c4...",
  "source_chunk_ids": ["chunk-abc123", "chunk-def456", ...],
  "entities_created": 12400,
  "relationships_created": 89000,
  "build_duration_seconds": 312.4
}
```

**Critical gap**: The current `--incremental` flag tracks which **chunk IDs** were processed, not which **file hashes** were current at build time. This means:
- A modified file generates the same chunk IDs (LanceDB replaces the row in-place), so the chunk-ID filter never detects the change
- Deleted files' entities are never purged from Kuzu
- The flag is effectively broken for true differential builds

---

## 2. Kuzu Schema — Exact Column Names

### CodeEntity node table (line 262-270)

```cypher
CREATE NODE TABLE IF NOT EXISTS CodeEntity (
    id STRING PRIMARY KEY,        -- = chunk_id from LanceDB
    name STRING,                  -- function/class/module name
    entity_type STRING,           -- "function", "class", "module", "struct", etc.
    file_path STRING,             -- relative path to source file (KEY FOR DIFFERENTIAL)
    commit_sha STRING,
    created_at TIMESTAMP DEFAULT current_timestamp()
)
```

`file_path` IS present on `CodeEntity`. This is the pivotal fact enabling file-scoped deletes.

### DocSection node table (line 281-292)

```cypher
CREATE NODE TABLE IF NOT EXISTS DocSection (
    id STRING PRIMARY KEY,
    name STRING,
    doc_type STRING,
    file_path STRING,             -- also has file_path
    level INT64,
    line_start INT64,
    line_end INT64,
    commit_sha STRING,
    created_at TIMESTAMP DEFAULT current_timestamp()
)
```

### Document node table (line 318-332)

```cypher
CREATE NODE TABLE IF NOT EXISTS Document (
    id STRING PRIMARY KEY,
    file_path STRING,             -- also has file_path
    title STRING,
    doc_category STRING,
    word_count INT64,
    section_count INT64,
    last_modified STRING,
    commit_sha STRING,
    created_at TIMESTAMP DEFAULT current_timestamp()
)
```

### Code relationship tables (lines 351-367)

All four code-to-code relationship types share the same schema:

```cypher
CREATE REL TABLE IF NOT EXISTS CALLS (
    FROM CodeEntity TO CodeEntity,
    weight DOUBLE DEFAULT 1.0,
    commit_sha STRING,
    MANY_MANY
)
-- Same schema for: IMPORTS, INHERITS, CONTAINS
```

### Other relationship tables

| Table | From → To |
|-------|-----------|
| FOLLOWS | DocSection → DocSection |
| REFERENCES | DocSection → CodeEntity |
| DOCUMENTS | DocSection → CodeEntity |
| HAS_TAG | Document/DocSection → Tag |
| DEMONSTRATES | DocSection → CodeEntity |
| LINKS_TO | DocSection → Document |
| CONTAINS_SECTION | Document → DocSection |
| RELATED_TO | DocSection → DocSection |
| DESCRIBES | DocSection → CodeEntity |
| HAS_TOPIC | Document → Topic |
| AUTHORED | Person → CodeEntity |
| MODIFIED | Person → CodeEntity |
| PART_OF | CodeEntity → Project |
| WRITTEN_IN | CodeEntity → ProgrammingLanguage |
| MODIFIES | Commit → CodeEntity |

### No `last_built` / `build_metadata` table in Kuzu

There is no timestamp column on entities and no metadata table in the Kuzu database itself. All metadata lives in the external `kg_metadata.json` file.

---

## 3. CLI Flags on `mvs kg build`

Both `build_kg` (line 557) and `index_kg` (line 606) commands in `kg.py` share these flags:

| Flag | Default | Description |
|------|---------|-------------|
| `--project-root` | `.` | Project root directory |
| `-f` / `--force` | False | Force rebuild even if graph exists |
| `--limit N` | None | Limit chunks processed (testing) |
| `--skip-documents` | False | Skip DOCUMENTS relationship extraction |
| `--incremental` | False | Only process new chunks not in previous build |
| `--verbose` / `-v` | False | Enable verbose debug output |

`--incremental` **already exists** as a flag stub. However, the current implementation is broken: it compares chunk IDs rather than file hashes, so modified files pass through undetected and deleted files are never purged.

The `_kg_subprocess.py` subprocess does **not** receive the `--incremental` flag — the parent process handles filtering before serialising the chunks JSON.

---

## 4. LanceDB Chunks Table — File-Hash Query Capability

### `CHUNKS_SCHEMA` (chunks_backend.py lines 51-88)

```python
CHUNKS_SCHEMA = pa.schema([
    pa.field("chunk_id", pa.string()),
    pa.field("file_path", pa.string()),
    pa.field("file_hash", pa.string()),    # SHA-256 of file content at index time
    # ... content fields ...
    pa.field("created_at", pa.string()),   # ISO timestamp
])
```

### Relevant existing methods in `chunks_backend.py`

| Method | What it does |
|--------|-------------|
| `get_all_indexed_file_hashes()` (line 598) | Returns `dict[file_path, file_hash]` for all indexed files — single full scan, O(1) per-file lookup after |
| `get_file_hash(file_path)` (line 640) | Returns stored hash for one file |
| `file_changed(file_path, current_hash)` (line 672) | Compares stored vs current hash |
| `delete_file_chunks(file_path)` (line 891) | Deletes all chunks for one file from LanceDB |
| `delete_files_batch(file_paths)` (line 933) | Batch-deletes chunks for multiple files |
| `iter_chunks_batched(file_path=None)` (lancedb_backend.py line 1061) | Streams chunks, optionally filtered by `file_path` |

**`kg_status` column does NOT exist yet** — the issue proposes adding it.

**No method to query chunks modified after a timestamp** — `created_at` is stored as a string, not used for incremental queries.

---

## 5. Option Evaluation

### Option A — File Hash Tracking via `kg_build_metadata.json`

Store `{file_path -> content_hash}` from the last KG build in `.mcp-vector-search/kg_build_metadata.json`. On incremental build:
1. Load stored hashes
2. Compute current hashes for all files via `get_all_indexed_file_hashes()` (single scan)
3. Identify: changed files (hash differs), deleted files (not in current set), new files (not in stored set)
4. Delete Kuzu entities for changed + deleted files
5. Re-process only changed + new files' chunks

**Correctness**: Handles deletes and renames correctly (deleted file's path disappears from LanceDB). Handles new files. Handles modifications. Cross-file relationship edges (e.g., A calls B, A changes) are handled by deleting all A's entities and edges, then re-inserting — B's incoming edges from A are re-created correctly. Global entities like `Person`, `Project`, `Repository` are rebuild-safe since they use upsert semantics.

**Risk**: The stored hashes diverge from reality if KG build fails mid-way. Must write metadata only after a fully successful build. Also, `module:*` entities (external library nodes with `file_path=""`) can never be individually targeted by file-scoped delete, but because they are upserted idempotently, this is harmless.

**Complexity**: Low. Requires:
- Add `file_hashes` dict to existing `kg_metadata.json` (no new file needed)
- Add `delete_entities_for_file(file_path)` to `KnowledgeGraph`
- Modify `_build_kg_impl` and `build_from_chunks_sync` to diff hashes and filter

### Option B — Kuzu `last_indexed_at` Timestamp on CodeEntity

Add a `last_indexed_at TIMESTAMP` column to `CodeEntity`. Query Kuzu for `MAX(last_indexed_at)`, then fetch LanceDB chunks with `created_at > that_timestamp`.

**Correctness**: Fails on file modification: LanceDB `created_at` is the timestamp when the chunk was first inserted, not when the file changed. When a file is re-indexed, its chunks are deleted and re-inserted — so the timestamp approach would catch re-insertions, but requires `created_at` to be updated on re-index (currently it is not always). More critically, **deleted files** are never surfaced by a timestamp query — a file that was removed has no chunks at all, so Kuzu entities would remain stale forever.

**Complexity**: High. Requires Kuzu schema migration (ALTER TABLE not supported in Kuzu — must recreate table). Adds timestamp tracking overhead.

**Risk**: High. Stale entities for deleted files. Timestamp drift if clock skew or batch processing re-uses timestamps.

### Option C — Reuse `chunks.lance file_hash`, Add `kg_status` Column

Add `kg_status: 'pending' | 'complete'` to `chunks.lance`. Store last KG build's `{file_path -> file_hash}` in metadata. On incremental:
1. Query LanceDB for all `(file_path, file_hash)` grouped by file
2. Compare against stored `file_hashes` in `kg_metadata.json`
3. Mark changed/new files' chunks as `kg_status = 'pending'`
4. Query `WHERE kg_status = 'pending'` and process only those
5. Mark processed chunks `kg_status = 'complete'`

This is what issue #108 proposes as the primary implementation.

**Correctness**: Handles all cases. The `kg_status` column is the source of truth, mirroring the existing `embedding_status` pattern. File deletes are detected by comparing the LanceDB file-path set against the stored set in metadata.

**Complexity**: Medium-high. Requires:
- LanceDB schema migration to add `kg_status` column to existing `chunks.lance` tables
- New `ChunksBackend` methods: `get_kg_pending_chunks()`, `mark_kg_complete(chunk_ids)`, `reset_all_kg_status()`
- Subprocess path must receive the pending chunk list (already serialised to JSON)
- Kuzu partial delete (same as Option A)

**Risk**: Schema migration is the main risk. LanceDB doesn't support `ALTER TABLE ADD COLUMN` natively — must use a data migration (read all → add column → overwrite). The `embedding_status` pattern exists and works, so the migration approach is proven.

---

## 6. Recommendation: Option A (Enhanced) with Option C's `kg_status` column as a follow-on

### Recommended Approach: Option A enhanced with file-hash metadata

**Rationale:**

1. **`file_path` is already on `CodeEntity`** — Kuzu delete-by-file-path is straightforward.
2. **`get_all_indexed_file_hashes()` already exists** — a single LanceDB scan returns all `{file_path -> file_hash}` pairs. No new LanceDB methods needed for change detection.
3. **No schema migration required** — just extend `kg_metadata.json` with a `file_hashes` dict. LanceDB schema is untouched.
4. **Option C's `kg_status` column** adds operational value (idempotent retry, visibility) but is a follow-on: it requires a migration and adds complexity without changing the correctness model. Implement Option A first, add `kg_status` in a subsequent PR.
5. **Option B is strictly worse** — timestamps don't detect deletes, and Kuzu schema migration is painful.

### Failure modes and mitigations

| Failure | Mitigation |
|---------|-----------|
| KG build fails mid-way | Write `kg_metadata.json` only after full success (already the pattern). Next run detects unchanged hashes → no spurious work. |
| `module:*` external entities accumulate | Acceptable: they are namespace-unique, idempotently inserted. Pruning is a separate concern. |
| Cross-file relationships (A calls B, A changes) | Delete all edges where source entity's `file_path = A`. Re-insert from fresh chunks. B's incoming edges from other files are unaffected. |
| Rename `a.py` → `b.py` | `a.py` disappears from LanceDB (deleted), `b.py` appears as new. Old `a.py` entities in Kuzu get deleted. New `b.py` entities are inserted. Correct. |
| KG was never built (first run) | `kg_metadata.json` doesn't exist → treat as full build. |
| Git entities (Person, Project, Repo) | These are always rebuilt in full at the end of `build_from_database`. No change needed. |

---

## 7. Concrete Implementation Plan

### Files to Modify

#### A. `src/mcp_vector_search/core/knowledge_graph.py`

Add method `delete_entities_for_file(file_path: str)`:

```python
def delete_entities_for_file(self, file_path: str) -> dict[str, int]:
    """Delete all KG entities and their relationships for a given file path.

    Kuzu does not support DETACH DELETE in the same syntax as Neo4j.
    The safe approach is:
      1. Collect entity IDs for the file.
      2. Delete all relationships where FROM or TO is one of those entities.
      3. Delete the entity nodes.

    Returns dict with counts: {"entities": N, "relationships": N}
    """
    counts = {"entities": 0, "relationships": 0}
    try:
        # Collect CodeEntity IDs for this file
        result = self.conn.execute(
            "MATCH (e:CodeEntity {file_path: $fp}) RETURN e.id",
            {"fp": file_path}
        )
        entity_ids = []
        while result.has_next():
            entity_ids.append(result.get_next()[0])

        if not entity_ids:
            return counts

        # Delete all code-to-code relationships touching these entities
        for rel_type in ["CALLS", "IMPORTS", "INHERITS", "CONTAINS"]:
            for eid in entity_ids:
                self.conn.execute(
                    f"MATCH (e:CodeEntity {{id: $id}})-[r:{rel_type}]-() DELETE r",
                    {"id": eid}
                )
                self.conn.execute(
                    f"MATCH ()-[r:{rel_type}]->(e:CodeEntity {{id: $id}}) DELETE r",
                    {"id": eid}
                )

        # Delete cross-type relationships (REFERENCES, DOCUMENTS, AUTHORED, etc.)
        for rel_type in ["REFERENCES", "DOCUMENTS", "DEMONSTRATES", "DESCRIBES",
                         "AUTHORED", "MODIFIED", "PART_OF", "WRITTEN_IN", "MODIFIES"]:
            for eid in entity_ids:
                try:
                    self.conn.execute(
                        f"MATCH (e:CodeEntity {{id: $id}})-[r:{rel_type}]-() DELETE r",
                        {"id": eid}
                    )
                    self.conn.execute(
                        f"MATCH ()-[r:{rel_type}]->(e:CodeEntity {{id: $id}}) DELETE r",
                        {"id": eid}
                    )
                except Exception:
                    pass  # Relationship type may not connect to CodeEntity

        # Delete the entity nodes
        self.conn.execute(
            "MATCH (e:CodeEntity {file_path: $fp}) DELETE e",
            {"fp": file_path}
        )
        counts["entities"] = len(entity_ids)

        # Also delete DocSection and Document nodes for this file
        self.conn.execute(
            "MATCH (d:DocSection {file_path: $fp}) DELETE d",
            {"fp": file_path}
        )
        self.conn.execute(
            "MATCH (d:Document {file_path: $fp}) DELETE d",
            {"fp": file_path}
        )
    except Exception as e:
        logger.warning(f"Failed to delete entities for {file_path}: {e}")

    return counts


def delete_entities_for_files(self, file_paths: list[str]) -> dict[str, int]:
    """Batch delete entities for multiple file paths."""
    total = {"entities": 0, "relationships": 0}
    for fp in file_paths:
        counts = self.delete_entities_for_file(fp)
        total["entities"] += counts["entities"]
        total["relationships"] += counts["relationships"]
    return total
```

**Note on Kuzu DELETE syntax**: Kuzu uses `DELETE node` (not `DETACH DELETE`). Relationship deletion must be done before node deletion. The above pattern collects IDs first, then deletes edge-by-edge. For large files with many entities, batch the edge deletes.

#### B. `src/mcp_vector_search/core/kg_builder.py`

Extend `_save_metadata()` to include `file_hashes`:

```python
def _save_metadata(
    self,
    source_chunk_count: int,
    source_chunk_ids: set[str],
    entities_created: int,
    relationships_created: int,
    build_duration_seconds: float,
    file_hashes: dict[str, str] | None = None,  # NEW
) -> None:
    metadata = {
        "last_build": datetime.now(UTC).isoformat(),
        "source_chunk_count": source_chunk_count,
        "source_chunk_id_hash": chunk_id_hash,
        "source_chunk_ids": list(source_chunk_ids),
        "entities_created": entities_created,
        "relationships_created": relationships_created,
        "build_duration_seconds": build_duration_seconds,
        "file_hashes": file_hashes or {},  # NEW: {file_path -> file_hash}
    }
```

Add new method `_get_changed_files()`:

```python
def _get_changed_files(
    self, current_file_hashes: dict[str, str]
) -> tuple[list[str], list[str], list[str]]:
    """Compare current file hashes against last-build hashes.

    Returns:
        (changed_files, new_files, deleted_files)
        All are lists of file_path strings.
    """
    metadata = self._load_metadata()
    if not metadata or not metadata.get("file_hashes"):
        # No previous build — treat everything as new
        return [], list(current_file_hashes.keys()), []

    stored_hashes: dict[str, str] = metadata["file_hashes"]

    changed = [
        fp for fp, h in current_file_hashes.items()
        if fp in stored_hashes and stored_hashes[fp] != h
    ]
    new_files = [
        fp for fp in current_file_hashes
        if fp not in stored_hashes
    ]
    deleted = [
        fp for fp in stored_hashes
        if fp not in current_file_hashes
    ]
    return changed, new_files, deleted
```

Modify `build_from_database()` differential logic:

```python
async def build_from_database(
    self, database, show_progress=True, limit=None,
    skip_documents=False, incremental=False,
) -> dict[str, int]:

    if incremental:
        # 1. Get current file hashes from LanceDB (single scan)
        current_hashes = await database.get_all_indexed_file_hashes()
        changed, new_files, deleted = self._get_changed_files(current_hashes)

        if not changed and not new_files and not deleted:
            logger.info("KG up to date — no file changes detected")
            return {}

        logger.info(
            f"Differential KG: {len(changed)} changed, "
            f"{len(new_files)} new, {len(deleted)} deleted files"
        )

        # 2. Delete Kuzu entities for changed + deleted files
        files_to_delete = changed + deleted
        if files_to_delete:
            self.kg.delete_entities_for_files(files_to_delete)

        # 3. Load chunks only for changed + new files
        files_to_process = set(changed + new_files)
        chunks = []
        for batch in database.iter_chunks_batched(batch_size=5000):
            filtered = [c for c in batch if str(c.file_path) in files_to_process]
            chunks.extend(filtered)
            if limit and len(chunks) >= limit:
                chunks = chunks[:limit]
                break

        if not chunks:
            logger.info("No chunks to process after filtering")
            # Still save metadata with updated hashes
            self._save_metadata(..., file_hashes=current_hashes)
            return {}

        # 4. Build from filtered chunks
        stats = await self.build_from_chunks(
            chunks, show_progress=show_progress, skip_documents=skip_documents
        )

        # 5. Save metadata with updated file hashes
        self._save_metadata(..., file_hashes=current_hashes)
        return stats
    else:
        # Full build — unchanged from current code
        ...
```

#### C. `src/mcp_vector_search/cli/commands/kg.py` — `_load_and_prepare_chunks()`

The parent-process chunk loader (lines 293-402) needs to perform the differential file-hash comparison **before** serialising chunks to the temp JSON. This is where `get_all_indexed_file_hashes()` is called (it needs the LanceDB connection).

Changes:
1. When `incremental=True`, call `database.get_all_indexed_file_hashes()` inside the `async with database:` block
2. Load and compare against `kg_metadata.json::file_hashes`
3. Identify `files_to_delete` and `files_to_process`
4. Pass `files_to_delete` to the subprocess (either as an env var JSON or a second temp file)
5. Filter the chunk iterator to `files_to_process`

Note: The subprocess receives chunks already pre-filtered. The Kuzu delete step must happen inside the subprocess (since KnowledgeGraph runs there). Pass `files_to_delete` as a JSON argument or temp file.

#### D. `src/mcp_vector_search/cli/commands/_kg_subprocess.py`

Add `--files-to-delete` argument:

```python
parser.add_argument(
    "--files-to-delete",
    type=str,
    default=None,
    help="Path to JSON file containing list of file_paths to delete from KG before build"
)
```

Before calling `builder.build_from_chunks_sync(...)`:

```python
if args.files_to_delete:
    with open(args.files_to_delete) as f:
        files_to_delete = json.load(f)
    if files_to_delete:
        console.print(f"[cyan]Deleting KG entities for {len(files_to_delete)} changed/deleted files...[/cyan]")
        kg.delete_entities_for_files(files_to_delete)
        console.print("[green]✓ Old entities deleted[/green]")
```

Also pass `file_hashes` in the subprocess call's metadata save so the parent can update `kg_metadata.json` after success. Alternatively, the parent writes metadata after subprocess exits with code 0.

#### E. `src/mcp_vector_search/cli/commands/index_cmd.py`

The `index_kg` command at line 286 (the primary `mvs index kg` path) needs the same differential logic. Check if it delegates to `_build_kg_impl` or has its own implementation and apply the same changes.

### New Data Structures

```python
# Added to kg_metadata.json
{
    ...,  # existing fields
    "file_hashes": {
        "src/mcp_vector_search/core/kg_builder.py": "a1b2c3d4...",
        "src/mcp_vector_search/core/knowledge_graph.py": "e5f6a7b8...",
        # one entry per indexed file
    }
}
```

```python
# New temp file passed to subprocess
# /tmp/kg_files_to_delete_XXXX.json
["src/old_module.py", "src/renamed_file.py"]
```

### New Methods Summary

| File | Method | Signature |
|------|--------|-----------|
| `knowledge_graph.py` | `delete_entities_for_file` | `(file_path: str) -> dict[str, int]` |
| `knowledge_graph.py` | `delete_entities_for_files` | `(file_paths: list[str]) -> dict[str, int]` |
| `kg_builder.py` | `_get_changed_files` | `(current_hashes: dict) -> tuple[list, list, list]` |
| `kg_builder.py` | `_save_metadata` | Add `file_hashes: dict | None` parameter |

No new LanceDB methods needed — `get_all_indexed_file_hashes()` already exists.

---

## 8. Implementation Sequence

1. **Kuzu delete method** (`knowledge_graph.py`) — implement and test in isolation with a small Kuzu database. Verify Kuzu's DELETE syntax for nodes with existing relationships.

2. **Metadata extension** (`kg_builder.py`) — add `file_hashes` to `_save_metadata` and `_get_changed_files`. Update `build_from_database` differential path.

3. **Subprocess wiring** (`_kg_subprocess.py`) — add `--files-to-delete` argument and pre-delete step.

4. **Parent process wiring** (`kg.py::_load_and_prepare_chunks`) — perform hash diff before chunk serialisation, pass `files_to_delete` to subprocess, pass `file_hashes` back for metadata save.

5. **Integration test** — index a small repo, verify KG build, modify one file, run `--incremental`, assert:
   - Modified file's old entities gone
   - Modified file's new entities present
   - Unchanged files' entities untouched
   - `kg_metadata.json::file_hashes` updated

6. **Follow-on: `kg_status` column** (Option C, per issue #108) — once Option A is stable, add `kg_status` to `chunks.lance` for better observability and retry semantics. This is a separate PR with a schema migration.

---

## 9. Key Risks and Open Questions

| Risk | Severity | Notes |
|------|----------|-------|
| Kuzu DELETE syntax for relationships | High | Must verify exact CQL syntax — no DETACH DELETE. Pattern: collect entity IDs, delete each relationship type explicitly, then delete nodes. |
| Large file sets (1000s of changed files) | Medium | Batch entity_id collection and deletion to avoid Kuzu memory issues. |
| `module:*` external entities with `file_path=""` | Low | Never targeted by file-scoped delete — safe, accumulate harmlessly. |
| `kg_metadata.json::file_hashes` size | Low | ~100 bytes per file × 39K files = ~3.9MB — acceptable. |
| Subprocess receives `files_to_delete` | Medium | Must use a second temp JSON file (same pattern as chunks JSON). Must clean up after subprocess exits. |
| `index_cmd.py` path | Medium | Needs same differential logic as `kg.py`. Confirm both code paths are updated. |

---

*Research saved to: `docs/research/differential-kg-build-design-issue-108-2026-03-09.md`*
*Issue: https://github.com/bobmatnyc/mcp-vector-search/issues/108*
