Metadata-Version: 2.4
Name: obsidian-agent-vault
Version: 0.2.0
Summary: Archive AI coding-agent sessions (Claude Code, Codex, Gemini, Cline, OpenCode, Hermes) as index-safe Markdown notes in an Obsidian vault.
Author: spawnofsociety2
License-Expression: MIT
Keywords: obsidian,markdown,claude-code,codex,ai-agents,transcripts,archiving
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Markup :: Markdown
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-subtests; extra == "dev"
Requires-Dist: coverage; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# agent-vault

[![PyPI version](https://badge.fury.io/py/obsidian-agent-vault.svg)](https://badge.fury.io/py/obsidian-agent-vault)

Archive your AI coding-agent sessions as clean, index-safe Markdown notes in an Obsidian vault.

If you use tools like Claude Code or Codex daily, your session history piles up in undocumented JSONL files and SQLite databases scattered across your machine. `agent-vault` sweeps those sources on a schedule and renders every session as a readable, searchable, wikilink-connected Markdown note — without ever freezing Obsidian's indexer, no matter how pathological the transcript.

## Why not just dump sessions to Markdown?

Because real agent transcripts are hostile to Obsidian. A single session can be 12 MB, contain 100,000-character physical lines, or embed NUL bytes — any of which can stall Obsidian's indexing for the whole vault. `agent-vault` was built after exactly that happened, and its pipeline is designed around preventing it:

- **Text normalization** — NULs removed, C0 control characters replaced, line endings normalized, physical line length capped (default 8,000 chars) with explicit continuation markers instead of silent truncation. Idempotent by construction.
- **Multipart splitting** — sessions are planned as a document set: one stable **primary note** plus `Part 001`, `Part 002`, … part notes, grouped by complete message sections against a 750 KiB target with a 1 MiB hard ceiling (`max_note_bytes`). Wikilinks always target the primary note.
- **Deterministic, idempotent sync** — a SQLite manifest tracks every generated file by content hash. Re-running an import produces zero writes. Interrupted runs converge on the next run. Files the tool didn't create are never touched or deleted.

## Supported providers

| Provider | Source format |
| --- | --- |
| Claude Code | `~/.claude/projects` JSONL session logs (including subagents) |
| Codex | `~/.codex/sessions` rollout logs |
| Gemini (Antigravity) | `~/.gemini/antigravity/brain` |
| Cline | `~/.cline` task history |
| OpenCode | `~/.local/share/opencode` session storage |
| Hermes | `~/.hermes` `state.db` SQLite databases |

Additional agents can be installed as third-party Python adapter packages
through the documented `agentvault.adapters` entry-point contract. Bundled
adapters always take precedence, conflicting plugins are rejected, and
`agent-vault --no-plugins <command> ...` disables plugin loading. See
[`docs/ADAPTERS.md`](docs/ADAPTERS.md#third-party-adapter-packages).

Multiple roots per provider are supported (e.g. a Windows path *and* a WSL `//wsl.localhost/...` path); identical sessions found in more than one place are deduplicated by content hash.

> **A note on fragility:** these session formats are undocumented and change without notice as the upstream tools evolve. The adapters were last validated against real session data in **July 2026**, specifically: Claude Code **2.1.201**, Codex CLI **1.17.15**, and Hermes Agent **0.18.0** (plus Gemini/Antigravity, Cline, and OpenCode session stores current at that date). If an import reports `failed` or `unsupported` sessions after a tool update, the adapter likely needs a refresh — failures print per-file diagnostics to stderr and are itemized in the `--report-json` output. Issues and PRs welcome.

## Install

Requires Python 3.11+. No runtime dependencies outside the standard library.

```bash
pip install obsidian-agent-vault
```

## Quickstart

1. Let `agent-vault` detect your installed agent stores and create a configuration:

   ```bash
   agent-vault init
   ```

   The guided flow shows every existing standard provider source directory it finds and asks for your vault path before writing `agent-vault.json`. For scripts and unattended setup, provide the choices explicitly:

   ```bash
   agent-vault init --vault /path/to/vault --yes
   ```

   Use repeatable `--source provider=/custom/path` arguments to add nonstandard locations, and `--no-detect` when you want only explicit sources. Existing configurations are never replaced unless you pass `--force`.

2. Validate the config without touching anything:

   ```bash
   agent-vault check --config agent-vault.json
   ```

3. Preview what an import would do — same planned paths and collision decisions as a real run, zero writes:

   ```bash
   agent-vault audit --config agent-vault.json --dry-run --report-json report.json
   ```

4. Run it for real:

   ```bash
   agent-vault audit --config agent-vault.json
   ```

Useful flags: `--provider <name>` scopes a run to one provider; `--full` re-imports sessions even if their sources appear unchanged; `--report-json <path>` writes run metrics.

Exit codes: `0` success, `1` source/output failures or unsupported providers, `2` invalid usage or config.

## Configuration

```json
{
  "vault": "/path/to/your/obsidian/vault",
  "sources": {
    "claude": ["~/.claude/projects"],
    "codex": ["~/.codex/sessions"],
    "hermes": ["~/.hermes"]
  },
  "redact_patterns": [],
  "max_tool_output": 1000,
  "max_note_bytes": 1048576,
  "max_line_chars": 8000
}
```

- `redact_patterns` — regex patterns scrubbed from all content before rendering and before content hashes are computed.
- `max_tool_output` — character cap for individual tool-output bodies.
- `max_note_bytes` — hard byte ceiling for any generated note (default 1 MiB).
- `max_line_chars` — maximum physical Markdown line length (default 8,000; minimum 80).

All limits are validated before any audit starts; `~` is expanded in all paths.

## What the output looks like

Notes land under `01_Transcripts/<Provider>/` in your vault with readable, date-prefixed, ID-disambiguated names:

```
01_Transcripts/Hermes/2026-06-30 Optimizing AI Setup (355c876a)/
├── 2026-06-30 Optimizing AI Setup (355c876a).md        ← primary (index) note
├── 2026-06-30 Optimizing AI Setup (355c876a) Part 001.md
├── ...
```

- Every note carries frontmatter: `provider`, `session_id`, `title`, `project`, `started_at`/`ended_at`, `source`, and tags.
- Sessions that spawned **subagents** get a directory, with child-session notes nested inside and two-way parent/child wikilinks.
- Multipart notes add `document_kind` (`primary`/`part`), `part`, and `part_count` — filter `document_kind != "part"` in Obsidian Bases/Dataview views so each session shows once.
- Small single-file sessions stay single files; the directory/part machinery only appears when needed.

## Scheduling

The CLI can install, inspect, and remove its own managed schedule:

```bash
agent-vault schedule install --config agent-vault.json --interval 6h --dry-run
agent-vault schedule install --config agent-vault.json --interval 6h
agent-vault schedule status
agent-vault schedule remove --dry-run
agent-vault schedule remove
```

`install` validates the configuration before touching the scheduler and
prints the exact Task Scheduler command or cron line it will use. Use
`--dry-run` to preview without changing the system. agent-vault manages only
its own single marked entry (`AgentVaultArchive` on Windows or
`# agent-vault:managed` in cron), and `remove` never touches unrelated work.
The initial automation supports Windows Task Scheduler and cron-compatible
systems; macOS `launchd` is not yet supported directly.

You can also author the scheduler entry yourself. A nightly run is one line:

```bash
# cron (Linux/macOS)
0 2 * * * agent-vault audit --config /path/to/config.json --report-json /path/to/nightly.json
```

```powershell
# Windows Task Scheduler action
agent-vault audit --config config.json --report-json nightly.json
```

Before scheduling, run once manually and open the vault in Obsidian to confirm indexing completes.

## Recovering from pathological notes

If a note (typically one generated before you adopted this tool, or by an older version) stalls Obsidian's indexer:

1. **Move it out** — don't delete it — to a quarantine folder *outside* the vault. Obsidian recovers immediately.
2. Re-run the importer scoped to that provider. The session regenerates from its original source under the current normalization and splitting rules.
3. Check the run report: `largest_note_bytes` and `largest_line_chars` should be within your configured ceilings, and generated files should contain no `U+0000`.
4. Delete the quarantined original only after confirming the regenerated note indexes cleanly.

**Finding leftovers:** sessions deduplicated against an already-imported copy are never re-rendered, so a stale pre-existing note for such a session won't be cleaned up automatically — the tool refuses to delete files it doesn't currently manage. Use `doctor` (below) to find and quarantine them.

## The doctor command

`doctor` scans everything actually on disk under `01_Transcripts/` — not just what the last run touched — and reports:

- **orphaned** — files that look like generated transcripts (frontmatter with `provider` and `session_id`) but that no manifest row owns: leftovers from older importer versions or deduplicated sessions. Hand-written notes are left alone.
- **oversized_line / oversized_note / nul_bytes** — violations of your configured index-safety limits, wherever they came from.

```bash
agent-vault doctor --config my-config.json --report-json doctor.json
# review the findings, then optionally:
agent-vault doctor --config my-config.json --quarantine-to /path/outside/vault/quarantine
agent-vault doctor --config my-config.json --quarantine-to /path/outside/vault/quarantine --only-unsafe
```

`--quarantine-to` moves **only orphaned files** (preserving their relative paths) — a hand-written note is never moved, even if it also trips `oversized_line` or another safety check, and manifest-owned files are never moved either, because the importer owns their lifecycle; regenerate those by re-running `audit`. Add `--only-unsafe` to narrow that further: only orphaned files that also carry a safety finding (`oversized_line`, `oversized_note`, or `nul_bytes`) get quarantined, so orphans that are merely harmless duplicates stay put for you to review. `--only-unsafe` requires `--quarantine-to` and exits `2` otherwise. Exit code is `1` when findings exist, `0` when clean, so it works as a scheduled health check.

## The reconcile command

`reconcile` compares what's still reachable from your configured sources against what the vault actually has, and reports drift — it never writes, moves, or deletes anything.

```bash
agent-vault reconcile --config my-config.json
agent-vault reconcile --config my-config.json --format json
```

Findings fall into exactly four categories:

- **stale-managed** — a manifest-owned note whose source session no longer exists at any configured source (deleted, moved, or its provider removed from the config).
- **content-duplicate** — two or more sessions, possibly under different providers or paths, whose normalized message content is identical.
- **identity-collision** — two different sessions that would resolve to the same vault note path.
- **orphan-unowned** — a file under `01_Transcripts/` that no manifest row owns, hand-written or otherwise (same ownership check `doctor` uses).

Every report opens with a line stating plainly that nothing was changed. Acting on a finding — regenerating a stale note, deleting one side of a duplicate, quarantining an orphan — is left to you; `reconcile` only reports. Exit code `1` means findings exist (useful for scheduled drift checks), `0` is clean, `2` is a config error.

## Optional AI enrichment

`agent-vault enrich` writes derived summary notes alongside your transcripts. It is entirely optional: nothing else in this project needs it, no runtime dependency is added, and no network code lives in the core. Enrichment shells out to an external command you configure — usually an AI CLI you already have installed — with a redacted transcript excerpt on that command's stdin, and captures its stdout as the summary.

```json
{
  "vault": "/path/to/your/obsidian/vault",
  "sources": { "claude": ["~/.claude/projects"] },
  "enrichment": {
    "command": ["claude", "-p"],
    "model_label": "claude-cli (claude -p), model unspecified by caller",
    "max_input_chars": 24000,
    "timeout_seconds": 120
  }
}
```

`command` is an argv list, not a shell string — no `{prompt}` templating, because the transcript excerpt is piped to the process's stdin rather than substituted into an argument. Both `claude -p` (no prompt argument, so it reads stdin) and `ollama run llama3` (reads stdin when stdin isn't a TTY) work this way. `model_label` is a free-text provenance string you choose — record whatever actually identifies what generated the note, since agent-vault has no way to verify it. `max_input_chars` (default 24,000) truncates the transcript excerpt sent to the command, with an explicit truncation marker rather than silent cutoff. `timeout_seconds` (default 120) bounds how long agent-vault waits for the command.

```bash
agent-vault enrich --config my-config.json               # generate/refresh sidecars
agent-vault enrich --config my-config.json --dry-run      # preview only, never invokes the command
agent-vault enrich --config my-config.json --force        # regenerate even if content is unchanged
agent-vault enrich --config my-config.json --session <id> # scope to one session
```

What you get, and the guarantees behind it:

- **Sidecar notes, never edits.** Output lands as new files under `enrichment/<primary-note-stem>.summary.md` — one per already-archived session. The transcript note under `01_Transcripts/` that a sidecar links to is never opened for writing.
- **Mandatory provenance frontmatter** on every sidecar: source session id and provider, a real Obsidian wikilink back to the primary transcript note, the `model_label` you configured, the exact command that was invoked, a content hash of the transcript text it summarized, and a generation timestamp. A sidecar's frontmatter tells you exactly what produced it and from what.
- **Idempotent by content hash.** Re-running `enrich` skips any session whose current transcript hash matches its existing sidecar's recorded hash — unchanged sessions are never re-summarized, so scheduled runs are cheap. Pass `--force` to regenerate regardless.
- **Only archives sessions `audit` has already written.** `enrich` looks up each session in the audit manifest so a sidecar's wikilink always points at the transcript note's real, on-disk filename; sessions not yet archived are skipped with a `skipped-not-archived` diagnostic (run `audit` first).
- **Failure-isolated.** A missing or failing command, a non-zero exit, empty output, or a timeout skips only that session with a diagnostic on stderr; nothing else is blocked or corrupted. Exit code `1` signals partial failure, `0` is clean, `2` is a config error (including a config with no `enrichment` section).
- **Enrichment is optional and derived** — see `CONTEXT.md`. It's never required to create or read an archive, `audit` never invokes it implicitly, and the `enrichment/` folder is safe to delete at any time; re-running `enrich` regenerates it from your still-intact transcripts.

## Development

See [`CONTRIBUTING.md`](CONTRIBUTING.md) for dev setup, lint/type-check commands, and PR expectations.

```bash
pip install -e ".[dev]"
python -m pytest tests/ -q
ruff check .
mypy
```

The suite covers adapters (with synthetic fixtures for every provider), text normalization, oversized-message chunking, document-set planning, manifest migration, transactional sync (including one↔many part transitions and interruption recovery), CLI behavior, and end-to-end index-safety regressions modeled on real failure shapes (12 MB transcripts, 136k-char lines, embedded NULs).

### Adapter compatibility and fixtures

See [`docs/COMPATIBILITY.md`](docs/COMPATIBILITY.md) for the provider compatibility matrix and what to do when an upstream format drifts, and [`docs/ADAPTERS.md`](docs/ADAPTERS.md) for how to author a provider adapter and its fixture/privacy rules. Two commands support that workflow without ever requiring a real transcript in the repo:

- `agent-vault fixture <provider> <source-file>` sanitizes one real session file into a structure-only fixture under `tests/fixtures/` (ids and paths become deterministic fake values, everything else becomes `REDACTED-<n>`; the source is only ever read). `tests/test_conformance.py` then runs every adapter against every fixture and fails with a clear message if a provider has none.
- `agent-vault report <path> [--output PATH]` prints a content-free structural diagnostic (provider detection, record-type histogram, key shapes, parse errors by line number) for attaching to bug reports instead of a transcript.

## License

[MIT](LICENSE)
