Metadata-Version: 2.4
Name: thinqos-harvest
Version: 0.29.3
Summary: Mind-as-observer probe - ship Claude Code (and other external) conversations into your thinqOS Mind.
Project-URL: Homepage, https://thinqos.com
Project-URL: Repository, https://github.com/AI4Outcomes/thinqos
Author-email: AI4Outcomes <support@thinqos.com>
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.13
Requires-Dist: click>=8.4
Requires-Dist: httpx>=0.28
Description-Content-Type: text/markdown

# thinqos-harvest

Mind-as-observer probe for [thinqOS](https://thinqos.com). Ships Claude Code and Codex sessions into your thinqOS Mind as captured Episodes, so the Mind learns from where you actually do your work - without sitting in the request path.

Pattern: **observer, not proxy.** The probe reads session JSONL from disk, normalizes it to the public [external ingest v1 contract](https://thinqos.com/docs/api/external-ingest/v1/schema.json), and POSTs it to `/api/ingest/external/v1` on your thinqOS instance. Claude Code and Codex can call it from Stop hooks; Codex Desktop, VS Code Codex, and Codex CLI can also be swept from the shared `~/.codex` session store for backfill.

## Install

The hook command needs [uv](https://docs.astral.sh/uv/) on PATH:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

Verify the install works:

```bash
uvx thinqos-harvest --help
```

## Wire up

One command makes thinqOS your default memory across Claude Code **and** Codex:

```bash
uvx thinqos-harvest install --pat tq_xxxx... --repair
```

`install` does five things, idempotently:

1. **Installs the tool** with `uv tool install thinqos-harvest` and wires hooks to
   the installed binary by absolute path - so a stale `uvx` cache never pins an
   old version, and a daily `uv tool upgrade` keeps it current.
2. **Wires the memory hooks** - **capture** (after each session), **incremental
   capture** (debounced mid-session snapshots at PostToolUse, so a mid-turn
   crash still lands the work), **reflexive retrieve** (before each turn),
   **resume** ("where you left off" at session start), and **self-update** (the
   daily background upgrade).
3. **Bootstraps your existing native memory** into the Mind in the **background** -
   the per-fact Claude memory files and the legacy Codex `MEMORY.md`, as durable
   native-memory Mind sources. The server queues extraction for the worker, so
   `install` spawns a detached drain that stores them in parallel and returns
   immediately; progress is logged to `~/.config/thinqos-harvest/import.log`. A
   content-hash manifest makes it resumable; server-side source IDs make imports
   idempotent; `--no-import-memory`
   skips it. Run `thinqos-harvest import-memory` anytime to drain or resume.
4. **Adds a write-redirect** so your agents persist *new* durable learnings into
   thinqOS: a one-line reminder in Claude's recall block, and a marker-delimited
   block in `~/.codex/AGENTS.md` (the only channel Codex honors). After this,
   native memory should contain only always-fire hard rules; thinqOS
   `reflexive_retrieve` + `recall_context` are the durable memory source of truth.
5. **Verifies the product contract** with `doctor`: managed hooks, Claude MCP
   registration, required Mind tools (`recall_context`, `mind_consult`, `observe`,
   `believe`), replay queue, server captures, installed binary, and auto-update
   status.

- **Mint a key** at <https://thinqos.com/api-keys> (*Create* -> copy the
  `tq_...`). Already connected the thinqOS MCP server? Omit `--pat` and the key
  is read from that registration.
- Scope to one client with `--client claude` or `--client codex` (default: both,
  whichever has a config directory).
- Add `--repair` when installing or re-running setup. It prints local health
  guidance and drains at most 25 queued replay payloads, so a stale offline queue
  cannot unexpectedly flush an unbounded backlog.
- **Safe to re-run.** Existing thinqOS hooks are detected, migrated, and never
  duplicated; your other hooks (coordinator, etc.) and AGENTS.md content are left
  untouched.
- Needs [uv](https://docs.astral.sh/uv/) on PATH (the same `uv`/`uvx` you used to
  run this command).

Open a new session (or restart your agent) and it will auto-capture, reflexively
retrieve your Mind/corpus, consult the Mind for high-stakes work, and greet you
with where you left off.

**Smoke test:** start a session, type one prompt, exit normally, then run
`uvx thinqos-harvest doctor` - it should show managed Claude hooks, the thinqOS
MCP registration, replay queue depth, and recent server captures without
printing your key. The final `health:` line should be `pass` or explain the
specific repair step.

## Repair, health, and uninstall

Repair is safe to run any time:

```bash
thinqos-harvest install --client claude --repair
thinqos-harvest doctor
```

`--repair` migrates stale managed hooks, refreshes the installed package path,
preserves unknown hand-authored hooks, and drains at most 25 queued replay
payloads so an old offline queue cannot unexpectedly flush without bounds.

Doctor is the local source of truth for install health:

```bash
thinqos-harvest doctor
```

It prints the installed version/binary, auto-update status, hook inventory, MCP
registration, required Mind tool availability, last hook status, replay queue by
source, and recent server captures. It ends with `doctor_checks:` and `health:`.

Uninstall removes local managed integration artifacts only:

```bash
thinqos-harvest uninstall --client claude
```

By default it removes managed Claude hooks and the `thinqos` Claude MCP
registration, preserves hand-authored hooks, preserves logs and queued replay
payloads under `~/.config/thinqos-harvest/`, and does **not** delete existing
server-side captures or Mind knowledge. Add `--dry-run` to preview changes,
`--remove-state` to delete local logs/queue/manifest, or `--remove-foreign-hooks`
only when you intentionally want to remove custom thinqOS hooks too.

## Fresh-machine continuity

On a second computer, run the same install command with an observer key for your
identity, restart Claude Code, and run `thinqos-harvest doctor`. thinqOS remains
the durable source of truth; local native memory is only for always-fire hard
rules. The session prime and resume hooks retrieve relevant Mind/project context
from thinqOS so Claude Code can continue from your accumulated history.

## Uninstall

Remove the managed local hooks with:

```bash
uvx thinqos-harvest uninstall --client both
```

Use `--client claude` or `--client codex` to remove only one integration. The
uninstaller removes hooks previously emitted by `thinqos-harvest install`,
recognized legacy thinqOS wrapper hooks, and the managed Codex write-redirect
block from `~/.codex/AGENTS.md`. It preserves unrelated hooks and unrecognized
hand-curated thinqOS wrapper commands so it does not destroy local automation
you wrote yourself.

This is local cleanup only. It does not delete server-side captures, revoke API
keys, or forget extracted knowledge. Use `thinqos-harvest forget <session_id>`
for captured sessions and revoke the observer key in thinqOS if the machine
should no longer connect. Add `--remove-tool` if you also want to run
`uv tool uninstall thinqos-harvest`.

### Manual wiring (advanced)

To hand-place the hooks instead, set `THINQOS_BASE_URL` and `INGEST_API_KEY` in
your shell rc, then run `uvx thinqos-harvest install-hook` (add
`--source openai.com/codex` for Codex). It prints a capture-only JSON snippet to
paste into `~/.claude/settings.json` (or `~/.codex/hooks.json`) under the
top-level `"hooks"` object, and warns rather than overwriting an existing `Stop`
hook.

## Codex

`install` already wires Codex. Codex Desktop, VS Code Codex, and Codex CLI write
rollout JSONL files under `~/.codex/sessions/YYYY/MM/DD/`; older files may live
under `~/.codex/archived_sessions/`. The Codex adapter reads both locations.
Codex passes the current `transcript_path` to the hook; the hook uploads the
completed turn and exits 0 so it does not block Codex.

For a one-time backfill of past Codex sessions, run:

```bash
uvx thinqos-harvest run --sources openai.com/codex --since 2026-05-01T00:00:00 --batch-size 5
```

Prefer a bounded backfill such as the last 7 days. `run` chunks uploads by
default so large local histories do not exceed server request limits.

For a one-shot run without installing the package globally:

```bash
THINQOS_BASE_URL=https://thinqos.com \
INGEST_API_KEY=tq_xxxx... \
uvx thinqos-harvest run --sources openai.com/codex --since 2026-05-01T00:00:00 --batch-size 5
```

Omit `--sources` to sweep all registered adapters.

## CLI

| Command | What it does |
| --- | --- |
| `thinqos-harvest install [--client ...] [--no-import-memory] [--repair]` | One-shot setup: install the tool, wire hooks, bootstrap legacy native memory, add the write-redirect. Safe to re-run; `--repair` drains up to 25 queued replay payloads. |
| `thinqos-harvest uninstall [--client ...] [--remove-tool] [--remove-state] [--dry-run]` | Remove managed local hooks/MCP config safely. Preserves hand-authored hooks and server-side captures by default; `--remove-tool` also removes the shared uv tool when no thinqOS hooks remain. |
| `thinqos-harvest import-memory [--concurrency N] [--dry-run] [--claude-limit N] [--codex-limit N]` | Queue legacy native Claude/Codex memory as durable Mind sources, most-recent first, `N` in parallel (default 5). Idempotent server-side and resumable by content-hash manifest; `0` limit = all pending. |
| `thinqos-harvest list [--limit N]` | List your captured sessions newest-first. |
| `thinqos-harvest forget <session_id>` | Delete one captured session by id (irreversible). |
| `thinqos-harvest install-hook [--source SOURCE]` | Print the JSON hook snippet for Claude Code or Codex (advanced manual wiring). |
| `thinqos-harvest run [--sources SOURCE] [--since TIMESTAMP] [--batch-size N] [--no-drain-pending]` | Manual sweep - discover any sessions not yet shipped and POST them in chunks. Use for bounded Codex or Claude Code backfill after a long offline period. |
| `thinqos-harvest hook capture [--source SOURCE]` | Hook entry point - reads `transcript_path` from stdin JSON. You should not run this directly; Claude Code and Codex hooks do. |
| `thinqos-harvest hook capture-incremental [--source SOURCE]` | Hook entry point - mid-session snapshot wired at PostToolUse, debounced per session (`THINQOS_INCREMENTAL_MIN_INTERVAL_S`, default 90s) and shipped non-final. Fail-open; do not run directly. |
| `thinqos-harvest hook self-update` | Hook entry point - daily-gated background `uv tool upgrade`. Wired at SessionStart; do not run directly. |
| `thinqos-harvest doctor` | Show pass/fail install health: version/binary, hook health, self-update status, replay queue depth by source, local log location, thinqOS MCP registration/tool health, last hook, and recent server captures. |

## Denylists

Two opt-out layers, both edited at `~/.config/thinqos-harvest/`:

- **`denylist.txt`** - newline-delimited substrings; any session whose `cwd` contains a substring is skipped entirely. Example: add `personal-taxes` to skip captures from `~/Documents/personal-taxes/`.
- **Content denylist** is hard-coded: turns containing `.env`, `api_key=…`, `sk-…`, or matching `(api_key|secret|password|token)=<16+ chars>` are dropped before POST. Oversized `tool_result` content (>32KB) is also dropped.

## Reliability

- Hook capture always exits 0; never blocks a Claude Code or Codex session even on bug or network failure.
- Hook health is written to `~/.config/thinqos-harvest/status.json`; detailed hook output goes to `~/.config/thinqos-harvest/hook.log`.
- Failed POSTs spool to `~/.config/thinqos-harvest/pending/<uuid>.json` and drain on the next `run` by default. Use `--no-drain-pending` for a tightly scoped backfill.
- Server-side dedup keyed by `(source, source_external_id)`; re-shipping the same session is idempotent. Claude Code captures include a monotonic `session_revision` so a later hook can replace an earlier partial capture.

## Forgetting a capture

```bash
thinqos-harvest forget <session_id>
```

Where `<session_id>` is the value `thinqos-harvest list` shows in the last column. This deletes the Episode on the server side and cascades to any extracted knowledge. For Codex captures, pass `--source openai.com/codex`.

For a per-turn / per-content scrub (rather than whole-session delete), contact support. It's a deliberate Phase-A non-goal but ship-able if asked for.

## Privacy posture

Opt-out, not opt-in. Adapter ships everything that isn't denied. If you'd rather have explicit opt-in per session, this probe is the wrong tool for you - the design decision is explicit in the [spec](https://github.com/AI4Outcomes/thinqos/blob/main/docs/superpowers/specs/2026-05-26-mind-observer-pattern-design.md).

## Versioning

Tagged `tools/thinqos-harvest-vX.Y.Z` in the thinqos monorepo. A push to a matching tag triggers the publish workflow.
