Metadata-Version: 2.1
Name: ctxforge-ai
Version: 0.1.0
Summary: Context compiler: raw documents to a queryable wiki (concepts, backlinks, search, health checks)
License: MIT
Keywords: ai,agents,context,rag,wiki,llm
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# ctxforge — compile raw knowledge into agent-ready context

The context-layer fix. Instead of wiring raw documents straight into an
agent's context window (slow, noisy, pattern-matched guesswork), ctxforge
*compiles* a source directory into a structured wiki: per-document summaries,
keyword lists, TF-IDF concept clusters with backlinks, a health-check linter,
and a small search engine an agent can query over a CLI.

## Why

- **Context is the bottleneck, not the model.** Karpathy: "a large fraction of
  my recent token throughput goes into manipulating knowledge" — raw sources
  compiled into a wiki with summaries, backlinks, and concept articles, queried
  through a CLI.
- **Vector retrieval breaks on vocabulary mismatch** ("why did the deploy fail"
  vs `get_pipeline_run_logs`). TF-IDF keyword retrieval over *compiled*
  summaries keeps agents on-target without an embedding service or vector DB.
- **Health checks catch decay**: stale sources, missing summaries, orphan docs
  — the wiki is treated as a product with an update cadence, not a setup step.

## Usage

```bash
./ctxforge compile ~/notes -o ~/wiki          # build wiki (extractive, $0, offline)
./ctxforge compile ~/notes -o ~/wiki --llm    # LLM-enriched summaries (optional)
./ctxforge search ~/wiki "reservoir pressure decline"   # ranked results
./ctxforge health ~/wiki                       # stale / orphan / missing-summary lint
CTXFORGE_AGENT_JSON=1 ./ctxforge search ~/wiki "deploy failed"  # JSON for agents
```

LLM enrichment is optional and model-agnostic (any OpenAI-compatible endpoint):

```bash
export OLLAMA_URL=http://localhost:11434/v1/chat/completions   # local
export CTXFORGE_LLM_URL=https://api.openai.com/v1/chat/completions
export CTXFORGE_LLM_KEY=sk-...
export CTXFORGE_LLM_MODEL=llama3.1
```

## Output

```
wiki/
  index.json        # machine-readable: docs, summaries, keywords, concepts, idf
  docs/<slug>.md    # one per source: summary + keywords + backlinks
  concepts/concept-N.md  # TF-IDF clusters with defining terms + source links
```

Search returns ranked paths + summaries; with `CTXFORGE_AGENT_JSON=1` it emits
a single JSON line an agent can consume directly.
