Metadata-Version: 2.4
Name: resonance-lattice
Version: 0.9.0
Summary: Portable project memory cartridges for AI assistants
Author: Kane Snyder
License: BSL-1.1
Project-URL: Repository, https://github.com/tenfingerseddy/Resonance-Lattice
Keywords: project-memory,ai-assistants,context-injection,retrieval,knowledge-cartridge
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.11
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.36
Requires-Dist: safetensors>=0.4
Requires-Dist: datasketch>=1.6
Requires-Dist: zstandard>=0.22
Requires-Dist: faiss-cpu>=1.7
Provides-Extra: stream
Requires-Dist: watchdog>=4.0; extra == "stream"
Provides-Extra: train
Requires-Dist: datasets>=2.16; extra == "train"
Requires-Dist: spacy>=3.7; extra == "train"
Requires-Dist: accelerate>=0.25; extra == "train"
Provides-Extra: bench
Requires-Dist: rank_bm25>=0.2; extra == "bench"
Requires-Dist: beir>=2.0; extra == "bench"
Requires-Dist: memory-profiler>=0.61; extra == "bench"
Provides-Extra: saas
Requires-Dist: fastapi>=0.115; extra == "saas"
Requires-Dist: uvicorn>=0.30; extra == "saas"
Requires-Dist: pydantic-settings>=2.0; extra == "saas"
Requires-Dist: PyJWT>=2.8; extra == "saas"
Requires-Dist: stripe>=8.0; extra == "saas"
Requires-Dist: azure-storage-blob>=12.19; extra == "saas"
Requires-Dist: azure-servicebus>=7.11; extra == "saas"
Requires-Dist: azure-cosmos>=4.5; extra == "saas"
Provides-Extra: fast
Requires-Dist: onnxruntime>=1.17; extra == "fast"
Provides-Extra: code
Requires-Dist: tree-sitter>=0.23; extra == "code"
Requires-Dist: tree-sitter-go>=0.23; extra == "code"
Requires-Dist: tree-sitter-rust>=0.23; extra == "code"
Requires-Dist: tree-sitter-typescript>=0.23; extra == "code"
Requires-Dist: tree-sitter-java>=0.23; extra == "code"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: hypothesis>=6.90; extra == "dev"
Requires-Dist: ruff>=0.2; extra == "dev"
Provides-Extra: all
Requires-Dist: resonance-lattice[bench,code,dev,fast,mcp,stream,train]; extra == "all"
Dynamic: license-file

# Resonance Lattice

**Portable semantic context for AI assistants.**
**Frontier retrieval without retrieval engineering.**

Build a local knowledge cartridge from your docs and code, then query, inspect, compare, and compose it as a single `.rlat` artefact.

```bash
pip install resonance-lattice
rlat build ./docs ./src -o project.rlat
rlat search project.rlat "how does this project work?" --format text
```

Resonance Lattice gives assistants grounded, evidence-backed project context without requiring a vector database, cloud service, or LLM. It runs fully locally, integrates with MCP-compatible assistants, and keeps the workflow stable even as backbone models evolve.

---

## What Is Resonance Lattice?

Resonance Lattice is a portable semantic knowledge layer for private knowledge and AI assistants. It turns a corpus of files into a single `.rlat` cartridge you can query, inspect, compare, and compose.

This is not a hosted retrieval service and not an embedding model. It is the layer that makes retrieval usable, portable, inspectable, and swappable.

`rlat build` chunks your files, encodes each chunk with a local encoder, and packages everything into a single `.rlat` cartridge. No LLM, no cloud service. The cartridge has three layers:

| Layer | What it does |
|-------|-------------|
| **Field** | Models semantic structure — what the corpus appears to know |
| **Registry** | Maps semantic hits back to ranked source files |
| **Store** | Returns evidence text, passages, and metadata |

When you query, your question is encoded the same way, resonated against the field, and the registry resolves the top matches back to source text. A search index answers queries. A semantic model can also describe its own shape, coverage gaps, and how it differs from another model.

You choose how the cartridge is packaged:

| Store mode | What you get | Trade-off |
|-----------|-------------|-----------|
| **embedded** (default) | Self-contained `.rlat` with field, registry, and all evidence text | Portable — share, archive, or move it anywhere |
| **external** | Lightweight `.rlat` with field and registry only; evidence read from local source files | Smaller cartridge, always up-to-date text, but tied to source directory |

---

## Why It Is Different

Most retrieval tools sell vector storage, embedding APIs, or managed RAG infrastructure. Resonance Lattice takes a different approach.

### 1. Your knowledge is a file

A cartridge is a single `.rlat` artefact you can move, version, archive, diff, and share like any other file. Not a service dependency.

### 2. It is mathematical, not generative

The field has algebraic structure. Every operation — adding a source, removing it, merging two cartridges, diffing versions — is a defined mathematical operation with known properties. Removing a file returns the field to the exact state it was in before that file was added. Merge is order-independent. The same inputs always produce the same field. No temperature, no sampling, no drift.

### 3. It is inspectable

You do not just get top-k results. Three commands expose the field's internal structure — insights no vector database can produce:

| Command | What it tells you |
|---------|------------------|
| `rlat xray corpus.rlat` | Corpus-level health: signal quality, saturation, diagnostic flags |
| `rlat locate corpus.rlat "query"` | Where a query sits in the knowledge landscape and what the field doesn't cover |
| `rlat probe corpus.rlat <recipe>` | Quick insight recipes: novelty, saturation, coverage gaps, and more |

### 4. It is composable -- fine-grained context control

Cartridges are building blocks, not monoliths. Build per-domain (docs, code, design, compliance) and compose per question. No index rebuild, no server restart, no cloud round-trip -- composition happens locally at query time.

No other retrieval system treats knowledge as an algebraic type.

| Operation | What it does | Guarantee |
|-----------|-------------|-----------|
| **merge** | Combine cartridges into a unified field | Commutative, associative |
| **diff** | Surface what changed between two snapshots | Directional signed delta |
| **forget** | Remove a knowledge subset cleanly | Algebraically precise |
| **project** | View one cartridge through the lens of another | Semantic projection, not keyword filter |
| **contradict** | Find where two cartridges disagree | Symmetric divergence detection |

```bash
# Multi-cartridge search
rlat search docs.rlat "auth flow" --with code.rlat

# View code through a compliance lens
rlat search code.rlat "data handling" --through compliance.rlat

# What changed since baseline?
rlat search current.rlat "what's new?" --diff baseline.rlat

# Boost security, suppress marketing
rlat search docs.rlat "overview" --boost "security" --suppress "marketing"

# Where do docs and code disagree?
rlat compose "docs.rlat ^ code.rlat" "authentication"

# Apply a knowledge lens
rlat search docs.rlat "user input" --with code.rlat --lens sharpen
```

The MCP tool `rlat_compose_search` exposes composition directly to coding assistants -- an LLM can merge, project, and diff cartridges inside a single tool call.

### 5. It is local-first

No cloud, no API keys, no LLM. The first `rlat build` downloads the encoder (~1.2 GB) from HuggingFace; after that, everything runs fully offline. Pair with a local LLM (Ollama, llama.cpp) for a fully private stack — your data never leaves your machine.

---

## Quick Start

```bash
pip install resonance-lattice
rlat build ./docs ./src -o project.rlat
rlat search project.rlat "how does auth work?" --format text
rlat profile project.rlat
```

Or set up the full assistant integration in one command:

```bash
rlat init-project --auto-integrate
# Builds cartridge, generates primer, creates manifest,
# updates .mcp.json, injects cartridge section into CLAUDE.md
```

Use the default embedded store for a self-contained cartridge, or `--store-mode external` for a smaller cartridge that reads evidence from local source files at query time.

For the full walkthrough — profiling, comparison, MCP integration, assistant context files, and HTTP serving — see [Getting Started](https://resonance-lattice.com/docs/getting-started).

---

## Assistant Integration

Resonance Lattice is built for assistant-native workflows. Three integration paths — pick the one that fits, or combine them. The MCP server is the fastest path for repeated use; the CLI is the universal fallback.

### MCP Server

The MCP server keeps your cartridge warm in memory and exposes it as native tools inside any MCP-compatible assistant. The assistant can search, profile, and compare your knowledge without leaving the conversation.

| Tool | Config file |
|------|------------|
| Claude Code | `.mcp.json` in project root |
| VS Code / GitHub Copilot | `.vscode/mcp.json` |
| Cursor | `cursor/mcp.json` or Settings > MCP |

```json
{
  "mcpServers": {
    "rlat": {
      "command": "rlat",
      "args": ["mcp", "project.rlat"]
    }
  }
}
```

This gives the assistant seven tools:

| Tool | What it does |
|------|-------------|
| `rlat_search` | Semantic query with ranked passages, band scores, and coverage |
| `rlat_info` | Cartridge metadata -- sources, bands, field dimensions |
| `rlat_profile` | Semantic shape: per-band energy, effective rank, spectral entropy, topic communities |
| `rlat_compare` | Compare the loaded cartridge against another |
| `rlat_compose_search` | Composed multi-cartridge search with projection, diff, boost/suppress, and per-cartridge injection modes |
| `rlat_discover` | List all available cartridges with domain, source count, and freshness status |
| `rlat_freshness` | Check if a cartridge is stale and needs rebuilding |

Queries hit the warm path (~80 ms) because the encoder and field stay loaded between calls.

### CLI

Any AI assistant with terminal access can call `rlat` directly — no configuration required beyond `pip install resonance-lattice`. This works with Claude Code, GitHub Copilot, Cursor, Codex, Cline, Windsurf, and any other tool that can run shell commands.

```bash
rlat search project.rlat "how does auth work?" --format json
rlat resonate project.rlat "what are the design constraints?" --format context
rlat profile project.rlat
```

You control how an LLM uses your knowledge with the `--mode` flag:

| Mode | What it tells the LLM |
|------|----------------------|
| **augment** | Use your own knowledge, but add detail and citations from these sources |
| **constrain** | Answer ONLY from the provided sources — if it's not covered, say so |
| **knowledge** | Base your answer primarily on this context; be transparent about gaps |
| **custom** | Your own system prompt, your rules |

```bash
rlat resonate project.rlat "how does auth work?" --mode constrain --format context
```

### Skills and Instruction Files

Skills today load the same static document regardless of the query. With cartridge-backing, a skill's context adapts to the specific request.

A skill becomes **cartridge-backed** by adding a few frontmatter fields:

```yaml
---
name: fabric-notebook-ingest
description: Create Fabric notebooks for data ingestion...
cartridges:
  - .rlat/fabric-docs.rlat
  - .rlat/pyspark-docs.rlat
cartridge-queries:
  - "How do you create a notebook in Fabric through the API"
  - "Fabric workspace authentication and authorization patterns"
cartridge-mode: augment
cartridge-budget: 2000
---
```

When a cartridge-backed skill triggers, context loads in up to four tiers:

| Tier | Source | What it provides |
|------|--------|-----------------|
| **1. Static** | SKILL.md header | Workflow structure, templates, decision trees |
| **2. Foundational** | Skill-authored queries | Baseline knowledge the skill always needs (40% of budget) |
| **3. Specific** | User query | Context unique to this request (30% of budget) |
| **4. Derived** | Caller-supplied queries | Implicit needs the user didn't express (30% of budget) |

Tiers 1-3 are automatic. Tier 4 accepts additional queries from the caller (e.g., LLM-generated search terms) via the `--derived` flag — this lets an orchestrating agent surface knowledge the user didn't know to ask for. Mode-aware gating controls injection: `augment` suppresses low-relevance context, `knowledge` uses a softer threshold, `constrain` always injects (zero-hallucination). Skills without `cartridge-*` fields work exactly as before. See [docs/SKILL_INTEGRATION.md](docs/SKILL_INTEGRATION.md) for the full architecture.

We also ship an **rlat skill** that routes conceptual questions through `rlat search` automatically and falls back to grep for exact symbol lookup. You can generate a **supplemental context file** and reference it from your instruction file:

```bash
rlat summary project.rlat -o .rlat/resonance-context.md
```

| Tool | Instruction file | Reference syntax |
|------|-----------------|------------------|
| Claude Code | `CLAUDE.md` | `@.rlat/resonance-context.md` |
| GitHub Copilot | `.github/copilot-instructions.md` | `@.rlat/resonance-context.md` |
| Cursor | `.cursorrules` | `@.rlat/resonance-context.md` |

### How It Complements Other Layers

| Layer | What it provides |
|-------|-----------------|
| Instruction files | Rules and project conventions (`CLAUDE.md`, `.cursorrules`) |
| Skills | Executable capabilities and tool-use patterns |
| Skills + Cartridges | **Adaptive context** — same skill, different knowledge per query |
| Memory | Persistent facts across conversations |
| **Resonance Lattice** | **Grounded project knowledge** — the actual content of your codebase and docs, semantically indexed |

Resonance Lattice fills the "what does this project actually contain" gap. It doesn't replace the other layers; it feeds them better context.

---

## Core Capabilities

### Search and Output Formats

The `--format` flag controls how results come back, so the same query serves different workflows:

| Format | What you get | Use it for |
|--------|-------------|-----------|
| **text** | Human-readable passages with source paths and coverage | Reading in the terminal |
| **json** | Structured object with scores, passages, metadata, and coverage | Scripts and programmatic consumption |
| **prompt** | Pre-formatted context block ready to paste into an LLM | Manual LLM workflows |
| **context** | Compressed context with injection framing | Automated assistant integration |

### Knowledge Algebra and Updates

Operate on knowledge artefacts directly: merge cartridges, diff versions, selectively forget sources. Three commands handle different update workflows:

| Command | What it does |
|---------|-------------|
| `rlat build` | Incremental by default — skips unchanged files via content hash |
| `rlat add` | Add new sources to an existing cartridge without touching anything else |
| `rlat sync` | Three-phase update: detect deleted files and remove them, then add new and updated files |

Removing a file is an exact algebraic operation — the field returns to the same state as if that file had never been added. No drift, no approximation.

### Use Cases

- **Grounding LLMs with context control** -- build per-domain cartridges, compose exactly the context an LLM needs per question. Use docs as augmentation, compliance as constraint.
- **Multi-team knowledge composition** -- engineering, legal, and product each maintain their own cartridges. Cross-functional questions compose all three at search time.
- **Compliance auditing** -- project code through a compliance cartridge to surface every point where implementation touches a regulated concern. Run contradiction detection to find policy/implementation drift.
- **Knowledge evolution tracking** -- diff successive builds to track how a codebase evolves. Search the delta directly: "what changed about authentication since last quarter?"
- **Role-based knowledge access** -- algebraic access control creates auditable, role-scoped views. Provable: visible + hidden = original.
- **Cross-domain discovery** -- cascade a query across cartridge boundaries. A security question starts in code, propagates through docs, surfaces a contradiction in compliance.
- **CLI semantic search** -- query docs and code without leaving the terminal
- **Coverage profiling** -- inspect what documentation covers and where it's thin

---

## Benchmarks

Resonance Lattice is not "a new embedding model." Its value comes from the portable cartridge, the retrieval pipeline, and the assistant workflow.

### Internal Retrieval

On a 24,635-chunk Microsoft Fabric documentation corpus with 100 evaluation questions:

| | rlat (reranked) | Hybrid RRF | Flat E5 | BM25 |
|---|---|---|---|---|
| **Recall@5** | **1.00** | 0.94 | 0.93 | 0.84 |
| **MRR** | **0.93** | 0.77 | 0.80 | 0.72 |
| **Failed retrieval** | **0%** | 6% | 7% | 16% |

### Token Efficiency And Grounding

Feeding rlat context to an LLM reduced hallucinations from 78% to 16% and lifted fact recall from 0.27 to 0.91. The LLM is still doing the thinking — but now it has something real to think about.

On a 2,266-file corpus, rlat search returns much more compact grounded context than reading files from keyword search:

| Approach | Tokens per query |
|----------|-----------------|
| grep + read top 5 files | 37,154 |
| rlat search (top 10) | 1,518 |

That is **24.6x fewer tokens** with ranked passages, source attribution, and semantic structure instead of raw file text.

### Cross-Corpus (BEIR)

Tested on 5 independent corpora the system has never seen, using the production encoder (E5-large-v2, no trained heads). Best mode per dataset shown:

| BEIR Dataset | rlat (best) | Mode | Flat E5 | BM25 |
|---|---|---|---|---|
| SciFact (5K) | **0.713** | reranked | 0.735 | 0.665 |
| NFCorpus (3.6K) | **0.360** | reranked | 0.337 | 0.325 |
| FiQA (57K) | **0.393** | reranked | 0.350 | 0.236 |
| ArguAna (8.7K) | **0.492** | dense | 0.501 | 0.315 |
| SciDocs (25K) | **0.189** | dense | 0.158 | 0.158 |

The pipeline exceeds flat E5 on 3 of 5 datasets (NFCorpus, FiQA, SciDocs). On ArguAna and SciDocs, dense-only outperforms reranking — the reranker helps factual/technical corpora but can hurt argument-style retrieval.

### What This Means

On corpora the system has never seen, the full pipeline outperforms flat E5 cosine similarity on 3 of 5 BEIR datasets — and comes within 97% on SciFact. The dense field with auto-mode selection (reranking when it helps, skipping when it doesn't) makes the pipeline adaptive across corpus types.

Results vary by dataset, and fit for your own corpus matters more than any single benchmark. The claim is not "best backbone." The claim is that Resonance Lattice gives you a stronger product shape for retrieval: portable, inspectable, local-first assistant context — with competitive cross-corpus numbers to back it up.

For full methodology and extended comparisons, see `benchmarks/results/`.

---

## How It Fits

| Approach | Strength | What rlat adds |
|----------|----------|----------------|
| **grep** | Exact text match | Semantic retrieval, 24.6x fewer tokens |
| **Standard RAG** | Cosine retrieval | Portable artefact, inspection, algebra |
| **LLM direct** | Reasoning + generation | Grounded evidence from your sources |
| **Obsidian vault** | Graph-linked notes | Automatic semantic indexing and evidence-backed search over existing files |

Resonance Lattice is not a replacement for any of these. It is the knowledge layer that makes each of them work better.

---

## Trade-offs and Boundaries

Every design choice has a cost. These are the ones worth knowing before you commit.

| What | Detail | Trade-off |
|------|--------|-----------|
| **Initial build is CPU-intensive** | First build encodes every chunk through E5-large-v2 | Incremental sync only re-processes changed files. ONNX runtime (`pip install onnxruntime`) provides 2-5x CPU speedup. CUDA GPU supported if available. |
| **English text only** | The default encoder is English-optimised. Non-English text encodes with degraded retrieval quality. | The backbone is configurable (`--encoder`), so a multilingual model can be substituted — but benchmarks need revalidation. |
| **Dense field caps around 82K sources** | The default dense field degrades beyond ~82K sources as interference accumulates. | Factored and product-quantised field backends handle larger corpora with reduced memory. |
| **Full pipeline required for some benchmark numbers** | The best retrieval numbers come from the full hybrid pipeline (lexical injection + reranking) on factual/technical corpora. On argument-style corpora, dense-only mode can outperform reranking. | Full pipeline adds ~800 ms cold / ~80 ms warm per query. Selective skip logic avoids the reranker when the top result is already confident. |

The important point: these are backbone and pipeline trade-offs, not workflow trade-offs. The `.rlat` cartridge remains the stable user-facing abstraction.

---

## Status

**Alpha** (`0.9.0`)

## Docs

- [Getting Started](https://resonance-lattice.com/docs/getting-started)
- [CLI Reference](docs/CLI.md)
- [Encoder Guide](docs/ENCODERS.md)
- [RQL Reference](docs/RQL_REFERENCE.md)
- [FAQ](https://resonance-lattice.com/docs/faq)

## License

[BSL-1.1](LICENSE.md) (converts to Apache 2.0 on 2029-04-09).
