Metadata-Version: 2.4
Name: athenaeum
Version: 0.8.0
Summary: Open source knowledge management pipeline — append-only intake, tiered compilation, configurable schemas
Project-URL: Homepage, https://github.com/Kromatic-Innovation/athenaeum
Project-URL: Repository, https://github.com/Kromatic-Innovation/athenaeum
Project-URL: Issues, https://github.com/Kromatic-Innovation/athenaeum/issues
Author-email: Kromatic Innovation <open-source@kromatic.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,compilation,knowledge-management,llm,memory
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: anthropic<1.0,>=0.39.0
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: pyyaml<7,>=6.0
Provides-Extra: bench
Requires-Dist: pytest-benchmark<6.0,>=4.0; extra == 'bench'
Requires-Dist: pytest>=8.0; extra == 'bench'
Provides-Extra: dev
Requires-Dist: fastmcp<4.0,>=2.0.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: fastmcp<4.0,>=2.0.0; extra == 'mcp'
Provides-Extra: vector
Requires-Dist: chromadb<2.0,>=0.5.0; extra == 'vector'
Description-Content-Type: text/markdown

# Athenaeum

[![PyPI version](https://img.shields.io/pypi/v/athenaeum.svg)](https://pypi.org/project/athenaeum/)
[![Python versions](https://img.shields.io/pypi/pyversions/athenaeum.svg)](https://pypi.org/project/athenaeum/)
[![License](https://img.shields.io/pypi/l/athenaeum.svg)](https://github.com/Kromatic-Innovation/athenaeum/blob/develop/LICENSE)

**Production-tested agentic memory for teams deploying multiple AI agents.**
Append-only intake, a tiered librarian that compiles raw observations into a
trustworthy wiki, and a sidecar that makes recall happen passively on every
turn.

<p align="center">
  <img src="docs/assets/athena.png" alt="Athena with her owl companion, holding an open book showing a knowledge graph" width="360">
</p>

> **Is this for me?** If you're running more than one agent on shared
> knowledge — or if you want agents and humans reading and writing the same
> institutional memory — yes. If you're building a single-user chatbot,
> [mem0](https://github.com/mem0ai/mem0) or
> [Letta](https://github.com/letta-ai/letta) may be a better fit.

## Why Athenaeum

Four design choices separate a production memory system from a single-user
markdown file. Each one fixes something that quietly breaks when a team scales
past one agent:

1. **[Sources as first-class objects](docs/why-athenaeum.md#1-sources-are-first-class-objects-trust-but-verify)** — every claim carries provenance, the way Wikipedia does. An unfootnoted fact is an assertion.
2. **[The librarian — a tiered compilation pipeline](docs/why-athenaeum.md#2-the-librarian--a-tiered-compilation-pipeline)** — agents can only _append_ to raw intake. A separate compiler is the only writer to the wiki. Safety from structure, not trust.
3. **[Passive recall](docs/why-athenaeum.md#3-passive-recall--recall-on-every-turn-automatically)** — a hybrid FTS5+vector search fires on every turn and injects breadcrumbs into context. The agent doesn't have to remember to look.
4. **[An editable observation filter](docs/why-athenaeum.md#4-the-notetaker--a-configurable-editable-observation-filter)** — what the agent saves is governed by a prompt you can read, edit, and audit. Not a black box.

Full rationale, comparison with alternatives (Claude memory, Anthropic's
memory tool, RAG, Karpathy's gist, mem0/Letta/Zep/Cognee), and the lessons
from running it on our own operations live in
[**docs/why-athenaeum.md**](docs/why-athenaeum.md). For the companion blog
post: [What We Learned Running Our Own Operations on Agentic
Memory](https://kromatic.com/blog/agentic-memory-in-production/).

## Installation

Requires Python 3.11+.

```bash
pip install athenaeum
```

## Quick start

```bash
# Initialize a knowledge directory
athenaeum init                  # default: ~/knowledge
athenaeum init --path ~/my-knowledge

# Run the librarian (compile raw intake → wiki entities).
# `athenaeum run` needs ANTHROPIC_API_KEY — use --dry-run to explore keyless.
# `run` and `status` operate on ~/knowledge by default (point elsewhere with
# `--path`; `run` also accepts `--knowledge-root` as the original spelling).
athenaeum run
athenaeum run --dry-run         # inspect without writing

# Check status
athenaeum status
```

Full run with custom paths and budgets (`--max-api-calls 200` here
deliberately lowers the per-run API budget below the default of 800 —
omit the flag to accept the default):

```bash
athenaeum run \
  --raw-root ~/knowledge/raw \
  --wiki-root ~/knowledge/wiki \
  --path ~/knowledge \
  --max-files 50 \
  --max-api-calls 200 \
  --verbose
```

## MCP memory server

Athenaeum ships an MCP server exposing `remember` and `recall` tools so AI
agents can write to raw intake and search the compiled wiki:

```bash
pip install 'athenaeum[mcp]'
athenaeum serve --path ~/knowledge

# Smoke test the round-trip without a live session
athenaeum test-mcp
```

**Claude Code integration.** Add to your MCP config and it auto-starts with
every session:

```bash
claude mcp add --scope user athenaeum -- athenaeum serve --path ~/knowledge
```

Example round-trip:

> **User:** Tristan's partner is Amanda; they met at Stanford GSB.
>
> *(Claude calls `remember(content="Tristan's partner is Amanda; they met at Stanford GSB.", source="claude-session")`)*
>
> A raw observation lands in `raw/claude-session/20260417T…-…md`. On the
> next `athenaeum run`, the pipeline compiles it into Tristan's wiki
> entity (under "Key Contacts") and Amanda's own entity if she doesn't
> exist yet. Later sessions can ask _"who is Amanda?"_ and `recall`
> returns the compiled page.

## Answering pending questions

When Tier 3 can't resolve an ambiguity or a principled contradiction, the
librarian escalates to `wiki/_pending_questions.md`. Each escalation lands
as a block like:

```markdown
## [2026-04-20] Entity: "Acme Corp" (from sessions/20240406T120000Z-aabb0011.md)
- [ ] Is Acme still Series A after the 2026 recapitalisation?
**Conflict type**: principled
**Description**: Prior wiki says Series A; the 2026-04 raw file implies Series B.
```

You resolve a question one of two ways — pick whichever fits your workflow:

### Option 1 — Edit the file directly

Flip `[ ]` to `[x]` on the checkbox line and type your answer below the
checkbox (above or below the conflict-type / description lines — either
works; the parser strips those metadata lines when extracting the answer):

```markdown
## [2026-04-20] Entity: "Acme Corp" (from sessions/20240406T120000Z-aabb0011.md)
- [x] Is Acme still Series A after the 2026 recapitalisation?

They closed Series B on 2026-03-12, led by Acme Growth Partners.
The 2026-04 raw file is correct; the prior wiki entry is stale.

**Conflict type**: principled
**Description**: Prior wiki says Series A; the 2026-04 raw file implies Series B.
```

### Option 2 — Use the MCP tool

For containerized agents that can't touch the filesystem, `athenaeum serve`
exposes two tools:

- `list_pending_questions()` returns unanswered blocks as JSON — each item
  carries a stable `id` derived from the header + question text.
- `resolve_question(id, answer)` flips the checkbox and writes the answer
  body under it. It does **not** archive on its own — archival runs on the
  next `ingest-answers` pass.

### Step 2 — ingest the answers

Either way, run:

```bash
athenaeum ingest-answers --path ~/knowledge
```

Each `[x]` block is rewritten as a raw intake file under
`raw/answers/{timestamp}-{entity-slug}.md` with frontmatter linking back
to the original source, then moved into
`wiki/_pending_questions_archive.md` (newest-first, append-only — answered
blocks are never deleted, only moved). The next `athenaeum run` picks the
raw file up like any other intake and folds the answer into the wiki
entity.

Re-running with no new `[x]` blocks is a no-op. Malformed blocks are
preserved in place and logged to stderr, so a corrupt single entry cannot
poison the rest of the file.

## Transparent sidecar (Claude Code hooks)

For a fully passive experience where Claude auto-recalls relevant context on
every prompt and saves observations without explicit commands, configure
Claude Code hooks:

1. Copy the example hooks from `examples/claude-code/` to your scripts directory.
2. Add hook entries to `~/.claude/settings.json` (see `examples/claude-code/settings-snippet.json`).
3. Add CLAUDE.md instructions for proactive memory (see `examples/claude-code/CLAUDE.md.example`).

This gives you:

- **Auto-recall** — an FTS5 index is built at session start (~300ms); each user message triggers a <50ms search that injects relevant wiki pages into context.
- **Auto-remember** — Claude proactively saves important facts without being asked.
- **Context checkpointing** — observations are saved before context-window compaction.

Full setup guide, smoke test, and environment-variable reference:
[`examples/claude-code/README.md`](https://github.com/Kromatic-Innovation/athenaeum/blob/develop/examples/claude-code/README.md).

## Integrations

- **Claude Code auto-memory** — bridge `~/.claude/projects/<scope>/memory/` into Athenaeum's `raw/` intake so the librarian can cluster, merge, and contradiction-check Claude Code's durable memory alongside other sources. See [`docs/integrations/claude-code.md`](docs/integrations/claude-code.md).
- **Contradiction detection** — pipeline overview, cross-scope modes, source-precedence taxonomy, configuration reference, and cost model for the auto-memory contradiction path. See [`docs/contradiction-detection.md`](docs/contradiction-detection.md).

## Vector search (optional)

Athenaeum supports a vector search backend (chromadb + `all-MiniLM-L6-v2`)
for semantic recall alongside the default FTS5 keyword backend. The recall
hook runs a **hybrid FTS5 + vector merge** when vector is configured —
each backend rescues a failure class the other has (short-query proper-noun
collisions for vector; no-lexical-overlap semantic queries for FTS5).

```bash
pip install 'athenaeum[vector]'
```

Enable it in `athenaeum.yaml`:

```yaml
search_backend: vector
```

Full walkthrough and the four invariants a future simplification must not
remove: [`docs/recall-architecture.md`](docs/recall-architecture.md).

## Query-topic extraction (optional)

`athenaeum query-topics "your prompt"` runs a Haiku classifier that returns
substantive topics and ignores meta-instructions:

```bash
$ athenaeum query-topics "Without calling any tools, quote the block about Return Path verbatim"
Return Path
```

The naive regex+stopword fallback returns
`block,calling,quote,return,tools,verbatim,without` — burying "Return Path"
behind meta-instruction tokens. The example recall hook uses `query-topics`
to rescue named-entity recall on instruction-heavy prompts and falls back
silently to the regex extractor if the API key or CLI is unavailable.

## Environment variables

The table below covers the common knobs. The exhaustive list — every env var,
yaml key, and CLI flag with its code default and precedence chain — lives in
[`docs/configuration.md`](docs/configuration.md).

| Variable | Required | Description |
|----------|----------|-------------|
| `ANTHROPIC_API_KEY` | Yes (unless `--dry-run`) | API key for Tier 2/3 LLM calls |
| `ATHENAEUM_CLASSIFY_MODEL` | No | Override Tier 2 model. Precedence: env > `models.classify` in `athenaeum.yaml` > default `claude-haiku-4-5-20251001` |
| `ATHENAEUM_WRITE_MODEL` | No | Override Tier 3 model. Precedence: env > `models.write` in `athenaeum.yaml` > default `claude-sonnet-4-6` |
| `ATHENAEUM_RESOLVE_MODEL` | No | Override the contradiction-resolver model (default: `claude-opus-4-7`) |
| `ATHENAEUM_RESOLVE_MAX_PER_RUN` | No | Cap resolver calls per ingest run (default: `250`, raised from 50 in #187) |
| `ATHENAEUM_MAX_API_CALLS` | No | Run-level API call budget for `athenaeum run`. Precedence: `--max-api-calls` CLI flag > env > `librarian.max_api_calls` in `athenaeum.yaml` > default `800`. Env `0` is valid and defers the entire intake (writes `wiki/_deferred_work.md` and logs the DEGRADED summary); the CLI flag rejects `0` |
| `ATHENAEUM_MAX_FILES` | No | Per-run intake batch size for `athenaeum run`. Precedence: `--max-files` CLI flag > env > `librarian.max_files` in `athenaeum.yaml` > default `50`. Env `0` is valid (defer-everything window); the CLI flag rejects `0` |
| `ATHENAEUM_BATCH_MODE` | No | Opt-in [Batch API](https://platform.claude.com/docs/en/build-with-claude/batch-processing) mode for `athenaeum run` (#236): tier-2/tier-3 calls are submitted as batches at a 50% token discount. Latency-tolerant — most batches finish within an hour, 24h worst case — intended for the nightly run. Precedence: `--batch-mode` / `--no-batch-mode` CLI flags > env > `librarian.batch_mode` in `athenaeum.yaml` > default off (`--no-batch-mode` forces the synchronous path even when env/yaml turn batch mode on) |
| `ATHENAEUM_RESOLVE_AUTO_APPLY` | No | Auto-apply high-confidence resolutions (default: `true`). See [`docs/auto-resolve.md`](docs/auto-resolve.md) |
| `ATHENAEUM_RESOLVE_AUTO_APPLY_THRESHOLD` | No | Confidence floor for auto-apply, in `[0.0, 1.0]` (default: `0.90`) |
| `ATHENAEUM_RESOLVE_FULL_BODY_TOKEN_CAP` | No | Per-side body cap for the resolver's full-body context, ~4 chars/token (default: `1500`; must be positive) |
| `ATHENAEUM_CROSS_SCOPE_MODE` | No | Cross-scope contradiction detection: `off` / `ancestor` / `similarity` / `both` (default: `ancestor`). See [`docs/contradiction-detection.md`](docs/contradiction-detection.md) |
| `ATHENAEUM_RESOLVED_SIMILARITY_THRESHOLD` | No | Cosine threshold for matching new detections against the resolved-decision log (default: `0.83`) |
| `ATHENAEUM_TIER4_DEDUP` | No | Dedupe pending-question escalations by source-memory pair (default: `true`; set `false`/`0`/`no`/`off` for legacy always-append) |
| `ATHENAEUM_CACHE_DIR` | No | Cache root for the librarian's embedding/cluster pass (default: `~/.cache/athenaeum`) |
| `ATHENAEUM_TOPIC_MODEL` | No | Override query-topic model. Precedence: env > `models.topic` in `athenaeum.yaml` > default `claude-haiku-4-5-20251001` |
| `ATHENAEUM_OP_KEY_PATH` | No | 1Password path for the session-start `ANTHROPIC_API_KEY` bootstrap (default: `op://Agent Tools/Anthropic API Key/credential`) |
| `ATHENAEUM_PQ_SNOOZE_HOURS` | No | Snooze TTL in hours for pending-questions surfacing (default: `24`; consumed by the `resolve-questions` skill) |
| `ATHENAEUM_PYTHON` | No | Python interpreter used by the example hooks (default: `python3`) |
| `AUTO_RECALL` | No | Per-turn recall on/off (hook shell env; overrides `athenaeum.yaml`'s `auto_recall`). Default: `true` |
| `SEARCH_BACKEND` | No | `fts5` or `vector` (hook shell env; overrides `athenaeum.yaml`'s `search_backend`). Default: `fts5` |
| `ATHENAEUM_HOOK_DEBUG` | No | Set to `1` to log vector-backend errors from `user-prompt-recall.sh` to stderr |

**Shell-env overrides.** `AUTO_RECALL` and `SEARCH_BACKEND` are read from the
shell environment after the hook sources `~/.cache/athenaeum/config.env`, so
exports in your shell profile beat the cached config. Intentional (lets you
A/B-test a backend without editing `athenaeum.yaml`), but it's the first
thing to check when the hook "ignores" a config change.

**Claude Code auth caveat.** Claude Code's own `CLAUDE_CODE_OAUTH_TOKEN` is
scoped to its inference endpoint, and the Anthropic Messages API rejects it
with `401 OAuth authentication is currently not supported`. The pipeline and
example hooks need a separate console API key — see
[`docs/recall-architecture.md`](docs/recall-architecture.md#anthropic_api_key-bootstrap-sessionstart)
for the 1Password bootstrap pattern.

## Configuration

Settings are resolved in the order **CLI flag > env var > `<knowledge_root>/athenaeum.yaml` > built-in default**, so a one-off shell export beats the yaml without requiring an edit. The canonical reference for every knob — librarian budgets, model selection, contradiction/resolver tuning, recall/search, and hook environment — is [`docs/configuration.md`](docs/configuration.md). As one example, the contradiction-resolver knobs live under a top-level `resolve:` block:

```yaml
resolve:
  model: claude-opus-4-7          # ATHENAEUM_RESOLVE_MODEL
  auto_apply: true                # ATHENAEUM_RESOLVE_AUTO_APPLY (default: true)
  auto_apply_threshold: 0.90      # ATHENAEUM_RESOLVE_AUTO_APPLY_THRESHOLD, [0.0, 1.0]
  full_body_token_cap: 1500       # ATHENAEUM_RESOLVE_FULL_BODY_TOKEN_CAP, per-side body cap (~4 chars/token)
```

When `auto_apply` is on and a proposal's confidence meets or exceeds `auto_apply_threshold`, the pending-question block is auto-flipped to answered with an `Auto-resolved: true` audit-trail tag. See [`docs/auto-resolve.md`](docs/auto-resolve.md) for the full lane, including how to disable, lower the threshold, or reverse an auto-resolution.

**Alternative model gateways.** All model calls go through the Anthropic SDK, which honors `ANTHROPIC_BASE_URL` — so a LiteLLM proxy or any Anthropic-compatible gateway can serve alternative models with zero code change. Only Claude models are first-party tested; see [`docs/configuration.md`](docs/configuration.md#alternative-model-gateways-anthropic_base_url) for the details and [#234](https://github.com/Kromatic-Innovation/athenaeum/issues/234) for multi-provider tracking.

## Data formats

**Raw intake** lives in `raw/{source}/*.md` with the naming convention
`{timestamp}-{uuid8}.md` (e.g., `20240406T120000Z-aabb0011.md`). Each file is
a plain markdown document containing observations, notes, or session
transcripts. The `{source}` directory identifies the origin (e.g.,
`sessions`, `imports`).

**Wiki entity pages** live in `wiki/` with YAML frontmatter:

```yaml
---
uid: a1b2c3d4
type: person
name: Alice Zhang
aliases: [Alice]
access: internal
tags: [active]
created: '2024-04-06'
updated: '2024-04-06'
---
```

Entities are indexed in `wiki/_index.md` grouped by type. Conflicts requiring
human review are appended to `wiki/_pending_questions.md`. Each run logs
token usage and estimated costs at the end.

**Degraded runs.** When a run exhausts its API call budget (see
`ATHENAEUM_MAX_API_CALLS` above), it writes a `wiki/_deferred_work.md`
manifest itemizing the raw files it did not process and ends with a
warning-level `Done (DEGRADED — budget exhausted)` summary line — the
machine-greppable signal that intake was deferred rather than completed.
The deferred files stay on disk and are picked up automatically by the
next run. The manifest is overwritten on every budget-tripped run and
cleared by the next clean run (full, merge-only, or cluster-only).
The cap is enforced at the entity-tier loop; merge-phase and re-resolve
calls count toward the budget but do not themselves stop the run, so a
merge-heavy run can overshoot the cap before enforcement kicks in.
A degraded run still exits `0` by default; pass `athenaeum run
--strict-budget` to make a budget-tripped run exit nonzero instead —
opt-in, for exit-code-based alerting.

## Known limitations

Athenaeum is pre-1.0. These trade-offs are intentional for the current
release line:

- **No retrieval benchmarks yet.** The hybrid-search claim rests on concrete
  failure modes (proper-noun collision, no-overlap semantic queries) and
  production use — not a published eval against mem0 / Letta / Zep /
  Cognee. If you need benchmarked recall@k on a closed corpus, pick a tool
  that publishes numbers. If you want a knowledge base that survives your
  tool choices, this is for you. PRs adding an eval harness are very
  welcome.
- **FTS5 index rebuilds are non-atomic and unlocked.** A shell hook and the
  librarian run rebuilding simultaneously can race; the window is small and
  single-user wikis do not hit it in practice, but hardened multi-writer
  safety remains future work. Workaround: don't invoke `athenaeum rebuild-index` and
  `athenaeum run` concurrently on the same `$KNOWLEDGE_ROOT`.
- **The `keyword` search backend is a scan-on-query fallback.** It reads
  every wiki page on every query; fine under ~1,000 entities, painful past
  that. Use `search_backend: fts5` (default in the CLI and hooks) for any
  non-trivial wiki. The keyword backend exists as a zero-dependency baseline
  for tests and bootstrap.
- **Tier 4 (human escalation) is a file, not a workflow.** Conflicts land in
  `wiki/_pending_questions.md`; you read it and decide. No PR-opening, no
  Slack integration, no UI — on purpose, for now.

## Development

```bash
git clone https://github.com/Kromatic-Innovation/athenaeum.git
cd athenaeum
pip install -e ".[dev]"

pytest tests/ -v
ruff check src/ tests/
```

## Branch flow

Athenaeum follows trunk-style development, with `develop` as the active
branch and `main` as the released-revision pointer:

- **`develop`** is the active development branch and the GitHub default. All
  pull requests target `develop`.
- **`main`** carries the most recent released revision. Release tags
  (`vX.Y.Z`) live on `main` and trigger the PyPI release workflow.

Most users should install via `pip install athenaeum` (above). To work from
source against the latest released revision instead of the active branch,
clone and check out the latest tag:

```bash
git clone https://github.com/Kromatic-Innovation/athenaeum.git
cd athenaeum
git checkout "$(git describe --tags --abbrev=0)"
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the full promotion flow.

## Getting help

Rolling this out on a team? Open an
[issue](https://github.com/Kromatic-Innovation/athenaeum/issues) or reach out
via [kromatic.com](https://kromatic.com/). We talk to teams working through
agent-memory rollouts often and are happy to point at whatever's useful.

## License

Apache 2.0 — see [LICENSE](https://github.com/Kromatic-Innovation/athenaeum/blob/develop/LICENSE).
