Metadata-Version: 2.4
Name: okforge
Version: 0.9.2
Summary: okforge: a local-first LLM knowledge-base engine
Project-URL: Repository, https://github.com/okforge/okforge
Project-URL: Homepage, https://github.com/okforge/okforge
Project-URL: Issues, https://github.com/okforge/okforge/issues
Author: designcomputer
License: Apache-2.0
License-File: LICENSE
Keywords: agents,ai,document,knowledge-base,llm,pageindex,rag,retrieval
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: click==8.4.0
Requires-Dist: json-repair==0.59.10
Requires-Dist: litellm==1.87.2
Requires-Dist: mcp==1.27.1
Requires-Dist: openai-agents==0.17.3
Requires-Dist: pageindex==0.3.0.dev1
Requires-Dist: portalocker==3.2.0
Requires-Dist: prompt-toolkit==3.0.52
Requires-Dist: python-dotenv==1.2.2
Requires-Dist: pyyaml==6.0.3
Requires-Dist: rich==15.0.0
Requires-Dist: trafilatura==2.0.0
Requires-Dist: watchdog==6.0.0
Provides-Extra: dev
Requires-Dist: mypy==1.15.0; extra == 'dev'
Requires-Dist: pytest-asyncio==1.3.0; extra == 'dev'
Requires-Dist: pytest==9.0.3; extra == 'dev'
Requires-Dist: ruff==0.9.7; extra == 'dev'
Requires-Dist: types-pyyaml==6.0.12.20260518; extra == 'dev'
Description-Content-Type: text/markdown

# okforge

A local-first LLM knowledge-base engine. Point it at your documents;
it builds an interlinked wiki — per-document summaries, cross-document
concept and entity pages, extracted images, and real page citations
back to source. The wiki is plain Markdown with YAML frontmatter,
readable in Obsidian or any editor, and queryable from a CLI, a chat
REPL, or any MCP client.

## Why

Most retrieval setups hand an LLM a pile of raw chunks and hope. okforge
instead compiles your sources into curated pages *ahead of time* —
concepts and entities that already synthesize what's spread across
many documents, each claim traceable back to a `(p. N)` citation in
the original source. That matters most for models with limited
context, including small models running entirely on your own hardware:
they don't have to reconstruct an answer from scratch every query, and
what they do say is checkable against a specific page, not just
plausible-sounding.

The output follows the [Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) —
typed frontmatter, relative links, a predictable directory layout — so
the wiki a KB produces is portable, not locked to okforge itself.

## Install

```bash
pip install git+https://github.com/okforge/okforge@main
```

## Quick start

```bash
mkdir my-kb && cd my-kb
okforge init          # scaffold raw/, wiki/, .okforge/  (--json for scripts)
okforge add paper.md  # ingest (pre-convert non-md/pdf inputs first)
okforge query "What does the paper conclude?"
okforge chat          # interactive REPL over the wiki
okforge list --json   # machine-readable state (also: status, okf-lint)
okforge describe "One line about this project."   # curated description
```

The query agent reads curated pages first, then drills for detail with
a built-in `grep_wiki` lexical search (locate-then-read) rather than
re-embedding everything. `okf-lint` checks a wiki bundle's OKF
conformance.

Configuration lives in `.okforge/config.yaml` (model, language, entity
types, …) and `~/.config/okforge/global.yaml` (KB registry, default
KB). The LLM endpoint is configured litellm-style — any
OpenAI-compatible server works, including a local llama.cpp instance.

### Ingesting scans and non-text documents

`okforge add` accepts Markdown, plain text, and PDF directly. Anything
else — docx, pptx, scanned pages, photo catalogs — needs converting to
Markdown first, by a tool that knows your material. For scanned pages
specifically, [**okforge-vision-ocr**](https://github.com/okforge/okforge-vision-ocr)
(`pip install okforge-vision-ocr`) is built for exactly this: one
vision-LLM call per page produces both a clean Markdown transcription
and extracted photos/figures, plus a sibling `<doc>.pages.json` page
array that `okforge add` reads directly for real `(p. N)` citations in
the compiled summaries — no separate wiring needed.

```bash
okforge-vision-ocr scanned.pdf raw/book.md   # OCR + photo extraction
okforge add raw/book.md                      # ingest, with page citations
```

It works against any OpenAI-compatible vision-language model (tuned
against a locally-hosted Qwen3.6-27B-MTP, but not tied to it).

### MCP server

`okforge mcp` starts an MCP server over the KB it's run against — same
resolution as every other command (cwd walk-up, or `--kb-dir`). Tools:
`query`, `grep_wiki`, `read_wiki_page`, `status`, and `read_topic` when
the KB has `topic_tree` enabled. Read-only — ingest stays a deliberate
CLI action.

Two transports, picked with `--transport`:

```bash
# stdio (default) — the client spawns this process itself
claude mcp add --transport stdio okforge -- okforge --kb-dir /path/to/kb mcp

# Streamable HTTP — this process listens on a socket for clients to connect to
okforge --kb-dir /path/to/kb mcp --transport http --port 8000
claude mcp add --transport http okforge http://127.0.0.1:8000/mcp
```

Works the same way with any MCP client that supports stdio or
Streamable HTTP.

**No authentication, on either transport.** stdio is safe by default
(it's just this process's own pipes, nothing listens on the network).
`--transport http` binds `127.0.0.1` by default for the same reason —
if you change `--host` or put it behind a reverse proxy, only do so
over a network you already trust end-to-end (SSH tunnel, VPN, an
authenticating proxy) — never a directly-exposed port. Anyone who can
reach it gets full read access to the KB, including `query` (which
runs your configured LLM on your behalf), with no login step.

### Topic tree (experimental, per-KB opt-in)

For knowledge bases that outgrow a flat concept list: set
`topic_tree: true` in `.okforge/config.yaml`, then run `okforge
reindex`. Existing concepts cluster into named `concepts/<topic>/`
directories, each with a `_topic.md` summary node; later ingests place
new concepts by tree descent, and queries gain a `read_topic`
navigation tool for browsing top-down instead of scanning a flat list.

## Wiki layout

```
wiki/
  index.md              # document + concept index
  summaries/<doc>.md    # per-document summary (page citations when available)
  concepts/<name>.md    # cross-document concept pages
  entities/<name>.md    # named people/places/organizations/works
  sources/<doc>.md      # ingested source text
  sources/<doc>.json    # per-page text + images (when page-aware)
  sources/images/<doc>/ # extracted images
  log.md                # append-only ingest log
```

## Roadmap

Planned, not yet built:

- **Web UI** — a native UI for browsing and querying a KB (search,
  read wiki pages, run `query`/`chat`) shipped with the engine itself.
- **Multiple LLM endpoints** — a KB config today points `model:` at one
  litellm-style endpoint; planned support for registering several
  (different hosts, different models) and choosing between them per
  call rather than being locked to one for the whole KB.
- **Parallel processing** — ingest and query are fully serial today
  (one `add` at a time, one generation call at a time); planned
  concurrency for page-level work and multi-document ingest, bounded
  by each endpoint's slot budget so throughput scales with the
  hardware actually available.

## Development

```bash
uv run --extra dev python -m pytest tests/   # test suite
uv run --extra dev ruff check okforge tests  # lint
uv run --extra dev ruff format okforge tests # format
```

## Origins

okforge began as a hard fork of [VectifyAI/OpenKB](https://github.com/VectifyAI/OpenKB),
in the spirit of Karpathy's LLM-wiki idea, and has since diverged
deliberately rather than tracking it — local-only by default, its own
document-conversion boundary, and OKF conformance as a first-class
goal rather than an incidental format.

## License

Apache-2.0. Portions originate from the upstream OpenKB project
(copyright the original authors); okforge-specific changes are
maintained at [okforge/okforge](https://github.com/okforge/okforge).
