Metadata-Version: 2.4
Name: obsidian-agent-vault
Version: 0.1.0
Summary: Archive AI coding-agent sessions (Claude Code, Codex, Gemini, Cline, OpenCode, Hermes) as index-safe Markdown notes in an Obsidian vault.
Author: spawnofsociety2
License: MIT License
        
        Copyright (c) 2026 spawnofsociety2
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Keywords: obsidian,markdown,claude-code,codex,ai-agents,transcripts,archiving
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Markup :: Markdown
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# agent-vault

[![PyPI version](https://badge.fury.io/py/obsidian-agent-vault.svg)](https://badge.fury.io/py/obsidian-agent-vault)

Archive your AI coding-agent sessions as clean, index-safe Markdown notes in an Obsidian vault.

If you use tools like Claude Code or Codex daily, your session history piles up in undocumented JSONL files and SQLite databases scattered across your machine. `agent-vault` sweeps those sources on a schedule and renders every session as a readable, searchable, wikilink-connected Markdown note — without ever freezing Obsidian's indexer, no matter how pathological the transcript.

## Why not just dump sessions to Markdown?

Because real agent transcripts are hostile to Obsidian. A single session can be 12 MB, contain 100,000-character physical lines, or embed NUL bytes — any of which can stall Obsidian's indexing for the whole vault. `agent-vault` was built after exactly that happened, and its pipeline is designed around preventing it:

- **Text normalization** — NULs removed, C0 control characters replaced, line endings normalized, physical line length capped (default 8,000 chars) with explicit continuation markers instead of silent truncation. Idempotent by construction.
- **Multipart splitting** — sessions are planned as a document set: one stable **primary note** plus `Part 001`, `Part 002`, … part notes, grouped by complete message sections against a 750 KiB target with a 1 MiB hard ceiling (`max_note_bytes`). Wikilinks always target the primary note.
- **Deterministic, idempotent sync** — a SQLite manifest tracks every generated file by content hash. Re-running an import produces zero writes. Interrupted runs converge on the next run. Files the tool didn't create are never touched or deleted.

## Supported providers

| Provider | Source format |
| --- | --- |
| Claude Code | `~/.claude/projects` JSONL session logs (including subagents) |
| Codex | `~/.codex/sessions` rollout logs |
| Gemini (Antigravity) | `~/.gemini/antigravity/brain` |
| Cline | `~/.cline` task history |
| OpenCode | `~/.local/share/opencode` session storage |
| Hermes | `~/.hermes` `state.db` SQLite databases |

Multiple roots per provider are supported (e.g. a Windows path *and* a WSL `//wsl.localhost/...` path); identical sessions found in more than one place are deduplicated by content hash.

> **A note on fragility:** these session formats are undocumented and change without notice as the upstream tools evolve. The adapters were last validated against real session data in **July 2026**, specifically: Claude Code **2.1.201**, Codex CLI **1.17.15**, and Hermes Agent **0.18.0** (plus Gemini/Antigravity, Cline, and OpenCode session stores current at that date). If an import reports `failed` or `unsupported` sessions after a tool update, the adapter likely needs a refresh — failures print per-file diagnostics to stderr and are itemized in the `--report-json` output. Issues and PRs welcome.

## Install

Requires Python 3.11+. No runtime dependencies outside the standard library.

```bash
pip install obsidian-agent-vault
```

## Quickstart

1. Copy `audit-config.example.json`, set `vault` to your Obsidian vault path, and remove any providers you don't use.

2. Validate the config without touching anything:

   ```bash
   agent-vault check --config my-config.json
   ```

3. Preview what an import would do — same planned paths and collision decisions as a real run, zero writes:

   ```bash
   agent-vault audit --config my-config.json --dry-run --report-json report.json
   ```

4. Run it for real:

   ```bash
   agent-vault audit --config my-config.json
   ```

Useful flags: `--provider <name>` scopes a run to one provider; `--full` re-imports sessions even if their sources appear unchanged; `--report-json <path>` writes run metrics.

Exit codes: `0` success, `1` source/output failures or unsupported providers, `2` invalid usage or config.

## Configuration

```json
{
  "vault": "/path/to/your/obsidian/vault",
  "sources": {
    "claude": ["~/.claude/projects"],
    "codex": ["~/.codex/sessions"],
    "hermes": ["~/.hermes"]
  },
  "redact_patterns": [],
  "max_tool_output": 1000,
  "max_note_bytes": 1048576,
  "max_line_chars": 8000
}
```

- `redact_patterns` — regex patterns scrubbed from all content before rendering and before content hashes are computed.
- `max_tool_output` — character cap for individual tool-output bodies.
- `max_note_bytes` — hard byte ceiling for any generated note (default 1 MiB).
- `max_line_chars` — maximum physical Markdown line length (default 8,000; minimum 80).

All limits are validated before any audit starts; `~` is expanded in all paths.

## What the output looks like

Notes land under `01_Transcripts/<Provider>/` in your vault with readable, date-prefixed, ID-disambiguated names:

```
01_Transcripts/Hermes/2026-06-30 Optimizing AI Setup (355c876a)/
├── 2026-06-30 Optimizing AI Setup (355c876a).md        ← primary (index) note
├── 2026-06-30 Optimizing AI Setup (355c876a) Part 001.md
├── ...
```

- Every note carries frontmatter: `provider`, `session_id`, `title`, `project`, `started_at`/`ended_at`, `source`, and tags.
- Sessions that spawned **subagents** get a directory, with child-session notes nested inside and two-way parent/child wikilinks.
- Multipart notes add `document_kind` (`primary`/`part`), `part`, and `part_count` — filter `document_kind != "part"` in Obsidian Bases/Dataview views so each session shows once.
- Small single-file sessions stay single files; the directory/part machinery only appears when needed.

## Scheduling

The tool is headless and cron-friendly. A nightly run is one line:

```bash
# cron (Linux/macOS)
0 2 * * * agent-vault audit --config /path/to/config.json --report-json /path/to/nightly.json
```

```powershell
# Windows Task Scheduler action
agent-vault audit --config config.json --report-json nightly.json
```

Before scheduling, run once manually and open the vault in Obsidian to confirm indexing completes.

## Recovering from pathological notes

If a note (typically one generated before you adopted this tool, or by an older version) stalls Obsidian's indexer:

1. **Move it out** — don't delete it — to a quarantine folder *outside* the vault. Obsidian recovers immediately.
2. Re-run the importer scoped to that provider. The session regenerates from its original source under the current normalization and splitting rules.
3. Check the run report: `largest_note_bytes` and `largest_line_chars` should be within your configured ceilings, and generated files should contain no `U+0000`.
4. Delete the quarantined original only after confirming the regenerated note indexes cleanly.

**Finding leftovers:** sessions deduplicated against an already-imported copy are never re-rendered, so a stale pre-existing note for such a session won't be cleaned up automatically — the tool refuses to delete files it doesn't currently manage. Use `doctor` (below) to find and quarantine them.

## The doctor command

`doctor` scans everything actually on disk under `01_Transcripts/` — not just what the last run touched — and reports:

- **orphaned** — files that look like generated transcripts (frontmatter with `provider` and `session_id`) but that no manifest row owns: leftovers from older importer versions or deduplicated sessions. Hand-written notes are left alone.
- **oversized_line / oversized_note / nul_bytes** — violations of your configured index-safety limits, wherever they came from.

```bash
agent-vault doctor --config my-config.json --report-json doctor.json
# review the findings, then optionally:
agent-vault doctor --config my-config.json --quarantine-to /path/outside/vault/quarantine
agent-vault doctor --config my-config.json --quarantine-to /path/outside/vault/quarantine --only-unsafe
```

`--quarantine-to` moves **only orphaned files** (preserving their relative paths) — a hand-written note is never moved, even if it also trips `oversized_line` or another safety check, and manifest-owned files are never moved either, because the importer owns their lifecycle; regenerate those by re-running `audit`. Add `--only-unsafe` to narrow that further: only orphaned files that also carry a safety finding (`oversized_line`, `oversized_note`, or `nul_bytes`) get quarantined, so orphans that are merely harmless duplicates stay put for you to review. `--only-unsafe` requires `--quarantine-to` and exits `2` otherwise. Exit code is `1` when findings exist, `0` when clean, so it works as a scheduled health check.

## Development

```bash
pip install pytest pytest-subtests
python -m pytest tests/ -q
```

The suite covers adapters (with synthetic fixtures for every provider), text normalization, oversized-message chunking, document-set planning, manifest migration, transactional sync (including one↔many part transitions and interruption recovery), CLI behavior, and end-to-end index-safety regressions modeled on real failure shapes (12 MB transcripts, 136k-char lines, embedded NULs).

## License

[MIT](LICENSE)
