document

Overview

ContextPull turns a folder of documents into something an LLM agent can pull from the way Claude Code pulls from a codebase: a small index that is always in context, and tools that return exact sections on demand. The model never receives content it did not ask for.

This set is the source of truth for what we are building. Read in order the first time.

Document What it answers
Architecture What the parts are, how data flows, where the boundaries sit
Design document Why it exists, goals and non-goals, principles, the decisions and the alternatives we rejected
System design Data model, section IDs, ingest pipeline, index construction, tool contracts, delivery into hosts, scale tiers, failure modes, security
Tool reference The five tools, argument by argument, with examples
Evaluation How we measure it with ragbisect, and what we publish, including losses
Roadmap Four milestones with acceptance criteria
Multi-language SDK plan How TypeScript, Go, Java and others get ContextPull without forking the logic
Store format contract The rules a reader in any language must follow
Testing Test layers, scale and quality results, bugs the campaign found
Embedding ContextPull Using the library inside your own product: ingest, query, access control, air-gap, versioning
Decision records One page per irreversible-ish decision

One-paragraph summary

A corpus is ingested once into a single SQLite file: documents, heading-aware sections with stable IDs, an FTS5 full-text index, and a cached one-line summary per document. A compact table of contents built from those summaries is delivered into the model's context. The model then calls search, read, grep and neighbours to fetch exactly the sections it decides it needs, verbatim, with IDs it can cite. The same core is exposed three ways: as a Python library for teams building their own client, as an MCP server for any open host, and as an embedding recipe for direct API use. Claude Code is the development client. ragbisect is the measuring instrument, and the agentic configuration goes in the same table as bm25, dense and hybrid.

Status

M1 to M3 built and measured; M4 largely built: PDF and Office ingest, hybrid search, HTTP transport, all five ragbisect shapes, TypeScript and Go readers and servers, a second corpus, and a complex-scenario test layer. Published: PyPI contextpull and ragbisect 0.1.0, npm contextpull 0.1.0 (the Office, Go and scenario work is on main, not yet released). Website: https://mi2arun.github.io/contextpull/. The first agentic benchmark rows were retracted and recomputed on 2026-09-14; see Testing.

Vocabulary