Metadata-Version: 2.4
Name: obsearch
Version: 0.1.0rc1
Summary: MCP server for Obsidian vaults: hybrid retrieval with a server-enforced privacy boundary
Project-URL: Homepage, https://github.com/boyscout99/obsearch
Project-URL: Repository, https://github.com/boyscout99/obsearch
Project-URL: Issues, https://github.com/boyscout99/obsearch/issues
Author: Tommaso Praturlon
License-Expression: MIT
License-File: LICENSE
Keywords: claude,markdown,mcp,obsidian,privacy,rag,retrieval,semantic-search
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.13
Requires-Dist: chromadb<2,>=1.0
Requires-Dist: fastembed<0.9,>=0.7
Requires-Dist: flashrank<0.3,>=0.2.10
Requires-Dist: httpx<1,>=0.27
Requires-Dist: mcp<2,>=1.9
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: rank-bm25<0.3,>=0.2.2
Description-Content-Type: text/markdown

# Obsidian Semantic Search MCP

[![CI](https://github.com/boyscout99/obsearch/actions/workflows/ci.yml/badge.svg)](https://github.com/boyscout99/obsearch/actions/workflows/ci.yml)

**Real retrieval for your Obsidian vault, as an MCP server — with private notes structurally unreachable by cloud AI.**

The package, the CLI, and the repo are all named `obsearch`.

Point Claude Code, Claude Desktop, Cursor, or any MCP client at your vault and get hybrid semantic + keyword search, wikilink/tag graph expansion, and cross-encoder reranking — not just file CRUD. No Obsidian plugin, no Obsidian running, no cloud services: it works on any folder of Markdown files.

## Why this instead of another Obsidian MCP server?

- **Bring your own agent.** Most Obsidian MCP servers wrap file operations; the "intelligence" stays locked in a plugin pane. This one gives *any* MCP client retrieval-engineering-grade search as a tool, so the agent you already use can work over your notes.
- **Agents that act, not just answer.** The intended workload isn't only Q&A — it's an agent proposing wikilinks, finding duplicate notes to merge, and fixing tags, using semantic search inside its own loop.
- **A server-enforced privacy boundary.** Mark notes private by folder, tag, or frontmatter flag; a `public`-profile server *cannot* reach them — not "filtered out", structurally absent (see [Privacy model](#privacy-model--threat-model)). Frontier-model intelligence over your vault, without your journal in the context window.
- **Retrieval quality as engineering.** Hybrid BM25 + dense retrieval merged with reciprocal-rank fusion, graph-aware expansion through wikilinks and shared tags, composite reranking (semantic signal, title match, graph proximity, backlinks, recency), then a FlashRank cross-encoder pass — vs. raw cosine similarity.
- **Headless and plain-markdown.** Works on a server, in CI, or on a vault that has never seen the Obsidian app.

## Quickstart (60 seconds)

Requires Python 3.13+ and [uv](https://docs.astral.sh/uv/).

**1. Create the config** at `~/.config/obsearch/config.toml`:

```toml
[vault]
path = "~/Documents/MyVault"

[privacy]
folders = ["journal", "people"]   # never served on the public profile
tags = ["private"]                # #private (and nested #private/…) notes
# frontmatter_flag = "private"    # notes with `private: true` (default)
```

**2. Build the index** (first run downloads a small ONNX embedding model into `~/.cache` — one-time, a few minutes; the vault itself never leaves your machine):

```bash
uvx obsearch index
```

**3. Register the server** with your MCP client. Claude Code:

```bash
claude mcp add obsearch -- uvx obsearch serve
```

Claude Desktop (`claude_desktop_config.json`), Cursor, and friends:

```json
{
  "mcpServers": {
    "obsearch": {
      "command": "uvx",
      "args": ["obsearch", "serve"]
    }
  }
}
```

**4. Ask.** *"Where did I write about burnout recovery?"* — the client calls `search_vault` and gets ranked passages with note paths, even when your notes never use the word "burnout".

## Tools

| Tool | What it does |
|---|---|
| `search_vault` | Hybrid semantic + keyword search; ranked passages with source paths. |
| `read_note` | Full note content plus parsed frontmatter and wikilinks. |
| `list_notes` | `path — title` listing, optionally scoped to a folder. |
| `note_links` | Backlinks, outgoing wikilinks, tags, and connected notes. |
| `index_status` / `reindex_vault` | Inspect and incrementally rebuild the index. |

The v1 tool surface is deliberately **read-only**: MCP clients like Claude Code already have file tools for editing; this server's job is finding and reading the right notes safely.

## Privacy model / threat model

**Claim:** a server started with the `public` profile cannot return a private note — its content, its title, or its path — through any tool, even if the note became private after the last index run.

How it's enforced, in layers:

1. **No private vectors exist.** Two separate index trees are built per vault (`public` and `full`), each with its own ChromaDB collection, BM25 state, manifest, *and embedding cache*. Private notes are never embedded into the public tree — there is nothing to leak from the vector store, which matters because embeddings themselves are invertible enough to be sensitive.
2. **One code path for note access.** Every tool reads notes through a single privacy-enforcing vault service; there are no side-channel filesystem reads in tool handlers.
3. **Serve-time revalidation.** Rules are re-evaluated against the note's *current on-disk state* on every request. Flag a note `private: true` and it disappears from search results, listings, and link graphs on the very next call — before any reindex. A stale index can never leak a newly-private note.
4. **Config lives outside the vault** (`~/.config`, XDG). Anything with vault-write access — including an MCP client editing your notes — cannot rewrite the privacy rules.
5. **Indistinguishability.** A private note answers exactly like a missing one (`Note not found`), so probing for existence teaches nothing.
6. **Paths are judged after resolution.** Every candidate file is resolved before it is read, and must still land inside the vault root. A note is named — and matched against your folder rules — by where it really lives, so a symlink cannot launder a private note under a public-looking path, and nothing outside the vault is ever indexed.

The `full` profile (`serve --profile full`) bypasses the rules for trusted local consumers — e.g. a fully local model that never leaves your machine.

**Out of scope:** this protects against what *the model is sent*, not against a compromised machine; anyone with local filesystem access can read the vault directly. And if you paste a private note into your client yourself, no server can help.

### The second door: agents with file tools

The privacy profile governs *this server's tools*. It cannot govern the rest of your agent. Point Claude Code at your vault as a working directory and its own `Bash` and `Read` tools will happily open the journal folder the MCP refuses to return — the profile was never in that path. Two tiers, depending on how much you care:

**Tier 0 — host, convenient (the [Quickstart](#quickstart-60-seconds) setup).** The server runs on stdio, spawned per session by your client. No daemon, no Docker, no open port. The privacy profile holds for every MCP tool call, and you close the second door with client-side deny rules (e.g. Claude Code's permission settings) plus not running the agent from inside the vault. Good enough for most people; enforcement is app-level and cooperative.

**Tier 1 — sandboxed, strict.** Put the trust boundary *between* the agent and the server. The server stays on the host, where it holds vault access and enforces the profile; the agent runs in a container with **no vault mount at all**. There is no second door to close, because the vault simply is not on the container's filesystem — `Bash` and `Read` find nothing, and the MCP endpoint is the only channel in. Indexing also stays on the host, so "indexing needs the whole vault" never reaches the sandbox.

```
HOST                                CONTAINER (sandboxed agent)
┌─────────────────────────┐         ┌──────────────────────────┐
│ vault (files)           │         │ Claude Code              │
│ obsearch serve          │◀──HTTP──│  no vault mount          │
│   --transport http      │  :9000  │  Bash/Read see nothing   │
│   --profile public      │         │  MCP is the only channel │
└─────────────────────────┘         └──────────────────────────┘
       host.docker.internal:9000
```

On the host:

```bash
uvx obsearch index            # one-time, trusted
uvx obsearch serve --transport http --profile public
```

Unlike Tier 0, this is a long-running background process you start yourself. It listens on `127.0.0.1:9000` and serves the MCP endpoint at `/mcp` (add `--transport sse` instead for the legacy `/sse` endpoint, for clients that only speak SSE).

In the container's `.mcp.json` — and **no vault volume** in your `docker-compose.yml`:

```json
{
  "mcpServers": {
    "obsearch": {
      "type": "http",
      "url": "http://host.docker.internal:9000/mcp"
    }
  }
}
```

The server accepts the `Host: host.docker.internal:9000` header out of the box; the SDK's DNS-rebinding guard rejects everything else, and `--allowed-host HOST:PORT` extends the list if your setup needs it.

A complete, working container — Compose file, image, and default-deny egress firewall — is in [`examples/tier1-sandbox/`](examples/tier1-sandbox/).

**Networking caveat — read this before deploying:**

- **macOS / Windows (Docker Desktop):** the default `--host 127.0.0.1` is reachable from the container via `host.docker.internal` and **not** exposed to your LAN. No auth needed. If your container runs a default-deny egress firewall, allow the resolved `host.docker.internal` address — it is the vpnkit gateway (`192.168.65.254`), not the bridge gateway, so a "trust the local /24" rule does not cover it.
- **Linux:** `127.0.0.1` is not reachable from a container. Bind `--host 0.0.0.0` and run the container with `--add-host=host.docker.internal:host-gateway`. **This exposes the port to your LAN**, and the endpoint currently has no authentication — firewall the port to the Docker bridge, or stay on Tier 0 until token auth ships.

## Configuration reference

```toml
[vault]
path = "~/Documents/MyVault"
# name = "MyVault"
# ignore_patterns = [".git", ".obsidian", ".trash"]

[privacy]
folders = ["journal", "people/*"]  # glob per path segment or full path
tags = ["private"]                 # matches nested tags (private/work)
frontmatter_flag = "private"       # `private: true` in frontmatter

[embeddings]
backend = "fastembed"              # bundled ONNX model, zero setup (default)
# backend = "ollama"               # local-first upgrade path
# model = "nomic-embed-text"       # backend-specific model override
# ollama_base_url = "http://localhost:11434"
# backend = "none"                 # keyword-only search
```

CLI: `obsearch index [--vault PATH] [--profile public|full]` builds the index (and downloads models on first run — never during `serve`, so MCP clients never time out on startup); `obsearch serve` runs the stdio server. Index data lives under `~/.local/share/obsearch/`, per vault and per profile.

## A note on local models

Findings from building this: small local models are good at *grounded Q&A over retrieved passages* and bad at *being the agent* — they choke on coding-agent system prompts and tool loops. The design encodes that split by staying a retrieval layer and **not generating answers**:

- **Claude-class clients** reason over `search_vault` + `read_note` themselves. Putting a smaller model in the middle only adds a slower, weaker reasoner between the passages and the client that was going to read them anyway.
- **Local models stay first-class for embeddings** — `[embeddings] backend = "ollama"` keeps indexing and retrieval entirely on your machine, which is the part of the pipeline that touches every note.

Fully local Q&A — generation that also never leaves your machine — is an application concern, not a retrieval one, and belongs in a layer built on top of this server.

## Contributing & license

MIT. Contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for expectations (personal-pace maintenance, and the test bar for anything touching the privacy boundary).
