Metadata-Version: 2.4
Name: thinqos-harvest
Version: 0.10.1
Summary: Mind-as-observer probe - ship Claude Code (and other external) conversations into your thinqOS Mind.
Project-URL: Homepage, https://thinqos.com
Project-URL: Repository, https://github.com/AI4Outcomes/thinqos
Author-email: AI4Outcomes <support@thinqos.com>
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.13
Requires-Dist: click>=8.4
Requires-Dist: httpx>=0.28
Description-Content-Type: text/markdown

# thinqos-harvest

Mind-as-observer probe for [thinqOS](https://thinqos.com). Ships Claude Code and Codex sessions into your thinqOS Mind as captured Episodes, so the Mind learns from where you actually do your work - without sitting in the request path.

Pattern: **observer, not proxy.** The probe reads session JSONL from disk, normalizes it to the public [external ingest v1 contract](https://thinqos.com/docs/api/external-ingest/v1/schema.json), and POSTs it to `/api/ingest/external/v1` on your thinqOS instance. Claude Code and Codex can call it from Stop hooks; Codex Desktop, VS Code Codex, and Codex CLI can also be swept from the shared `~/.codex` session store for backfill.

## Install

The hook command needs [uv](https://docs.astral.sh/uv/) on PATH:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

Verify the install works:

```bash
uvx thinqos-harvest --help
```

## Wire up

One command makes thinqOS your default memory across Claude Code **and** Codex:

```bash
uvx thinqos-harvest install --pat tq_xxxx...
```

`install` does four things, idempotently:

1. **Installs the tool** with `uv tool install thinqos-harvest` and wires hooks to
   the installed binary by absolute path - so a stale `uvx` cache never pins an
   old version, and a daily `uv tool upgrade` keeps it current.
2. **Wires the memory hooks** - **capture** (after each session), **recall**
   (before each turn), **resume** ("where you left off" at session start), and
   **self-update** (the daily background upgrade).
3. **Imports your existing native memory** into the Mind - the per-fact Claude
   memory files and the Codex `MEMORY.md` - via `observe`. Run interactively, it
   shows how many facts are pending and asks how many of the **most recent** of
   each source to imprint now (default 150 each; Enter accepts, `0` skips a
   source); re-run anytime to drain the rest. `--yes` accepts the defaults,
   `--no-import-memory` skips it, and a content-hash manifest means re-runs only
   import new/changed facts.
4. **Adds a write-redirect** so your agents persist *new* durable learnings into
   thinqOS: a one-line reminder in Claude's recall block, and a marker-delimited
   block in `~/.codex/AGENTS.md` (the only channel Codex honors).

- **Mint a key** at <https://thinqos.com/api-keys> (*Create* -> copy the
  `tq_...`). Already connected the thinqOS MCP server? Omit `--pat` and the key
  is read from that registration.
- Scope to one client with `--client claude` or `--client codex` (default: both,
  whichever has a config directory).
- **Safe to re-run.** Existing thinqOS hooks are detected, migrated, and never
  duplicated; your other hooks (coordinator, etc.) and AGENTS.md content are left
  untouched.
- Needs [uv](https://docs.astral.sh/uv/) on PATH (the same `uv`/`uvx` you used to
  run this command).

Open a new session (or restart your agent) and it will auto-capture, auto-recall,
and greet you with where you left off.

**Smoke test:** start a session, type one prompt, exit normally, then run
`uvx thinqos-harvest list` - your session should appear within a second.

### Manual wiring (advanced)

To hand-place the hooks instead, set `THINQOS_BASE_URL` and `INGEST_API_KEY` in
your shell rc, then run `uvx thinqos-harvest install-hook` (add
`--source openai.com/codex` for Codex). It prints a capture-only JSON snippet to
paste into `~/.claude/settings.json` (or `~/.codex/hooks.json`) under the
top-level `"hooks"` object, and warns rather than overwriting an existing `Stop`
hook.

## Codex

`install` already wires Codex. Codex Desktop, VS Code Codex, and Codex CLI write
rollout JSONL files under `~/.codex/sessions/YYYY/MM/DD/`; older files may live
under `~/.codex/archived_sessions/`. The Codex adapter reads both locations.
Codex passes the current `transcript_path` to the hook; the hook uploads the
completed turn and exits 0 so it does not block Codex.

For a one-time backfill of past Codex sessions, run:

```bash
uvx thinqos-harvest run --sources openai.com/codex --since 2026-05-01T00:00:00 --batch-size 5
```

Prefer a bounded backfill such as the last 7 days. `run` chunks uploads by
default so large local histories do not exceed server request limits.

For a one-shot run without installing the package globally:

```bash
THINQOS_BASE_URL=https://thinqos.com \
INGEST_API_KEY=tq_xxxx... \
uvx thinqos-harvest run --sources openai.com/codex --since 2026-05-01T00:00:00 --batch-size 5
```

Omit `--sources` to sweep all registered adapters.

## CLI

| Command | What it does |
| --- | --- |
| `thinqos-harvest install [--client ...] [--no-import-memory]` | One-shot setup: install the tool, wire hooks, import native memory, add the write-redirect. Safe to re-run. |
| `thinqos-harvest import-memory [--dry-run] [--claude-limit N] [--codex-limit N] [--no-include-claude] [--no-include-codex]` | Imprint native Claude/Codex memory into your Mind via `observe`, most-recent first. Idempotent (content-hash manifest); resumable; `0` limit = all pending. |
| `thinqos-harvest list [--limit N]` | List your captured sessions newest-first. |
| `thinqos-harvest forget <session_id>` | Delete one captured session by id (irreversible). |
| `thinqos-harvest install-hook [--source SOURCE]` | Print the JSON hook snippet for Claude Code or Codex (advanced manual wiring). |
| `thinqos-harvest run [--sources SOURCE] [--since TIMESTAMP] [--batch-size N] [--no-drain-pending]` | Manual sweep - discover any sessions not yet shipped and POST them in chunks. Use for bounded Codex or Claude Code backfill after a long offline period. |
| `thinqos-harvest hook capture [--source SOURCE]` | Hook entry point - reads `transcript_path` from stdin JSON. You should not run this directly; Claude Code and Codex hooks do. |
| `thinqos-harvest hook self-update` | Hook entry point - daily-gated background `uv tool upgrade`. Wired at SessionStart; do not run directly. |
| `thinqos-harvest doctor` | Show hook health, replay queue depth, local log location, and recent server captures. |

## Denylists

Two opt-out layers, both edited at `~/.config/thinqos-harvest/`:

- **`denylist.txt`** - newline-delimited substrings; any session whose `cwd` contains a substring is skipped entirely. Example: add `personal-taxes` to skip captures from `~/Documents/personal-taxes/`.
- **Content denylist** is hard-coded: turns containing `.env`, `api_key=…`, `sk-…`, or matching `(api_key|secret|password|token)=<16+ chars>` are dropped before POST. Oversized `tool_result` content (>32KB) is also dropped.

## Reliability

- Hook capture always exits 0; never blocks a Claude Code or Codex session even on bug or network failure.
- Hook health is written to `~/.config/thinqos-harvest/status.json`; detailed hook output goes to `~/.config/thinqos-harvest/hook.log`.
- Failed POSTs spool to `~/.config/thinqos-harvest/pending/<uuid>.json` and drain on the next `run` by default. Use `--no-drain-pending` for a tightly scoped backfill.
- Server-side dedup keyed by `(source, source_external_id)`; re-shipping the same session is idempotent. Claude Code captures include a monotonic `session_revision` so a later hook can replace an earlier partial capture.

## Forgetting a capture

```bash
thinqos-harvest forget <session_id>
```

Where `<session_id>` is the value `thinqos-harvest list` shows in the last column. This deletes the Episode on the server side and cascades to any extracted knowledge. For Codex captures, pass `--source openai.com/codex`.

For a per-turn / per-content scrub (rather than whole-session delete), contact support. It's a deliberate Phase-A non-goal but ship-able if asked for.

## Privacy posture

Opt-out, not opt-in. Adapter ships everything that isn't denied. If you'd rather have explicit opt-in per session, this probe is the wrong tool for you - the design decision is explicit in the [spec](https://github.com/AI4Outcomes/thinqos/blob/main/docs/superpowers/specs/2026-05-26-mind-observer-pattern-design.md).

## Versioning

Tagged `tools/thinqos-harvest-vX.Y.Z` in the thinqos monorepo. A push to a matching tag triggers the publish workflow.
