Metadata-Version: 2.5
Name: deyta-cli
Version: 0.6.0
Summary: A unified command-line tool for Deyta's services, wrapping Khora persistent memory for AI agents.
Project-URL: Homepage, https://github.com/DeytaHQ/deyta-cli
Project-URL: Repository, https://github.com/DeytaHQ/deyta-cli
Project-URL: Issues, https://github.com/DeytaHQ/deyta-cli/issues
Author-email: "AllTheData Inc. (Deyta)" <dev@deyta.ai>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,ai,cli,embeddings,khora,memory,rag,typer
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Utilities
Requires-Python: >=3.13
Requires-Dist: fastapi>=0.115.0
Requires-Dist: httpx>=0.28.0
Requires-Dist: khora[embedded]>=0.26.0
Requires-Dist: mcp>=1.27.0
Requires-Dist: packaging>=24.0
Requires-Dist: questionary>=2.1.1
Requires-Dist: rich>=15.0.0
Requires-Dist: textual>=0.80.0
Requires-Dist: tomli-w>=1.0.0
Requires-Dist: typer>=0.26.7
Requires-Dist: uvicorn[standard]>=0.34.0
Provides-Extra: test
Requires-Dist: pytest-asyncio>=0.25; extra == 'test'
Requires-Dist: pytest>=8.0; extra == 'test'
Description-Content-Type: text/markdown

# Deyta CLI

A unified command-line tool for Deyta's services. Today it wraps
[Khora](https://github.com/DeytaHQ/khora) (persistent memory for AI agents); the
command surface is built so future services and a cloud platform slot in without
breaking existing commands.

## How it works

Khora is an in-process Python library, not a server. The CLI runs a local **daemon**
(`deyta serve` — a FastAPI app holding one `Khora` instance open) and talks to it over
HTTP. The CLI itself never imports Khora. The target server is resolved per command:

```
--host flag  >  DEYTA_HOST env  >  active context  >  http://localhost:8787
```

That resolution is the local↔cloud seam: switching to a cloud platform later is a new
context, not new commands.

## Requirements

- Python 3.13+
- `DEYTA_OPENAI_API_KEY` in `~/.config/deyta/.env` (Khora uses it for embeddings and entity extraction; `deyta init` prompts for it, or `deyta init --yes` picks it up from the environment instead of prompting)
- Docker — only for the `postgres` backend (`deyta db up`); the embedded backend needs none

## Install

`deyta` is a CLI, so install it as an isolated **tool** rather than into a project
environment. This puts the `deyta` command on your PATH and keeps its dependencies
from colliding with anything else:

```bash
uv tool install deyta-cli      # recommended
# or:
pipx install deyta-cli
```

Both create a dedicated environment just for Deyta and expose `deyta` everywhere — no
virtualenv to activate. To upgrade later: `uv tool upgrade deyta-cli` (or
`pipx upgrade deyta-cli`).

**One-line install (curl):**

If you'd rather not pick a tool, this script does it for you:

```bash
curl -fsSL https://raw.githubusercontent.com/DeytaHQ/deyta-cli/main/install.sh | sh
```

It picks an installer in order: **uv** if present (uv also fetches a Python 3.13
runtime, so you don't need a matching Python first); else **pipx** if you already have
it *and* Python 3.13+; otherwise it **asks before installing uv** (and aborts with
instructions if you decline) — it never modifies your system silently. Pin a version
with `DEYTA_VERSION=0.2.0`, or set `DEYTA_YES=1` to skip the uv prompt in CI.

Prefer to read before you pipe to a shell? Download, inspect, then run:

```bash
curl -fsSLO https://raw.githubusercontent.com/DeytaHQ/deyta-cli/main/install.sh
less install.sh        # review it
sh install.sh
```

This is a convenience wrapper, not a separate channel — it installs the same PyPI
package as `uv tool` / `pipx` below.

**macOS (Homebrew):**

```bash
brew tap deytahq/deyta
brew trust deytahq/deyta    # current Homebrew requires trusting any third-party tap
brew install deyta
```

This installs into its own virtualenv (using prebuilt wheels for the native
dependencies) and puts `deyta` on your PATH. Upgrade with `brew upgrade deyta`.

> The `brew trust` step is a Homebrew default for **all** non-official taps, not
> something specific to Deyta — without it Homebrew refuses to load the formula.

> Avoid `pip install deyta-cli`. Bare `pip` installs into whatever Python environment
> happens to be active, so the `deyta` command only works while that environment is
> activated — and it can clash with other packages. Use `uv tool` / `pipx` for CLIs.

### From source (development)

```bash
git clone https://github.com/DeytaHQ/deyta-cli && cd deyta-cli
uv sync                        # creates .venv with all deps
uv run deyta --help            # run without activating the venv
```

## Quickstart (embedded, no Docker)

```bash
deyta init                 # choose "embedded (sqlite_lance, no Docker)"
deyta init --yes           # or scripted: embedded backend + default models, no prompts
deyta up                   # start the whole stack in the background (datastores if postgres, then the daemon)

deyta ns create demo       # create a namespace; becomes active
deyta ingest run ./docs    # walk files, chunk + remember (sub-batched, live progress)
deyta query "your question"

deyta down                 # stop the stack when you're done
```

`deyta up` is the one-command path; `deyta serve` still exists if you'd rather run the
daemon in the foreground (and `deyta db up` to manage just the datastores).

### Non-interactive setup

`deyta init --yes` runs the wizard with no prompts — every unanswered question takes its
default. Flags override individual answers (each provided flag skips its own prompt even
without `--yes`), and the API keys are read from the environment rather than prompted for.
It prints a resolved-config summary so you can see exactly what it chose. Handy in CI,
containers, and agent-driven setups:

```bash
DEYTA_OPENAI_API_KEY=sk-… deyta init --yes                 # embedded backend, default models
deyta init --yes --backend postgres --llm-model gpt-4o     # postgres, custom LLM
```

API keys are never accepted as flags (secrets don't belong in argv). Under `--yes`,
`deyta init` reads `DEYTA_OPENAI_API_KEY` from the environment (or the global `.env`)
and persists it; a missing key is a warning, not an error. The server API key is not
handled by `init` — it is generated on first `deyta up` / `deyta serve`.

## Commands

| Command | Purpose |
|---|---|
| `deyta init [--yes]` | Scaffold `deyta.toml` (pick backend, ontology defaults); `--yes` for non-interactive/scripted setup |
| `deyta serve [--port] [--detach]` | Start the daemon (foreground by default) |
| `deyta status` / `deyta stop` | Inspect / stop a detached daemon |
| `deyta up` / `deyta down` | Bring the whole local stack up/down (datastores + server) |
| `deyta db up\|down\|status\|logs` | Manage Postgres + Neo4j (Docker; postgres backend) |
| `deyta ns create\|list\|get\|delete\|use\|stats` | Manage namespaces (`namespace` is the long form); `stats` shows document/chunk/entity/relationship counts |
| `deyta memory remember\|recall\|forget` | Khora primitives (`memory ingest` mounts the ingest group below) |
| `deyta ingest run <path>... [--expertise-file <yaml/json>]` | Bulk-ingest files/dirs with sub-batching, retry, and a resumable log (see [custom expertise](#custom-extraction-expertise---expertise-file)) |
| `deyta ingest status --log <path>...` | Summarize progress from one or more logs (offline; never contacts the server) |
| `deyta query "<text>"` | Shorthand for `memory recall` (`--mode`, `-k`, `--json`, `--context`, `--filter`) |
| `deyta config` | Interactive TUI editor for `deyta.toml` settings |
| `deyta config get\|set <key>` | Read/write a single config value (dot notation, e.g. `llm.model`) |
| `deyta config path` | Print the resolved config file path |
| `deyta context use\|list\|current` | Switch between local and cloud contexts |
| `deyta context add <name> --host <url> --api-key <key>` | Connect to a deployment someone else created |
| `deyta context remove <name>` | Remove a context (does not destroy the deployment) |
| `deyta deploy fly [--name] [--vm-size] [--vm-memory] [--volume-size]` | Deploy to Fly.io (create or redeploy) |
| `deyta deploy scale` | Resize the deployed machine/volume in place (no image rebuild) |
| `deyta deploy config` | Apply the deployment's config to the running app (no image rebuild) |
| `deyta deploy destroy` | Tear down the Fly app and all its data |
| `deyta login` / `logout` | Cloud auth (not yet available) |
| `deyta version [--no-check]` | Show installed CLI + Khora versions; flag PyPI updates |
| `deyta update [--yes] [--dry-run]` | Upgrade whichever of the CLI / Khora is outdated |

### Namespaces

A namespace is a memory silo. Khora keys them by UUID; the friendly name is stored on
the namespace server-side and mapped back to its UUID in `deyta.toml`, so commands can
say `demo` instead of a UUID.

No two *active* namespaces may share a name — `deyta ns create demo` is refused with a
conflict while a live `demo` exists, because a name that resolves to two namespaces
sends memory to whichever one the lookup happened to reach first. A blank name is
refused outright. The check reads the table and then creates, so it is not a hard
guarantee under genuinely concurrent creates; it settles the case that happens in
practice, which is reusing a name after deleting it.

`deyta ns delete` is a **soft delete**: the namespace is deactivated, not erased. Its
documents stay in storage, it drops out of `deyta ns list` and out of every memory
operation, and the name it held becomes free for a new namespace.

Should a store still hold two active namespaces of the same name (created before this
rule), `deyta ns use <name>` refuses to guess **when the name is not already mapped in
`deyta.toml`** — it prints the candidate UUIDs so you can pick one. A name already
mapped locally resolves to its pinned UUID and is used as-is, with no server lookup, so
the ambiguity check only runs for a name that isn't cached yet; address a specific
namespace by its UUID with `deyta ns use <uuid>`.

### Remembering a single note

`deyta memory remember "<text>"` stores one piece of text. Alongside `--title` and
`--source`, two optional flags attach an external reference and opaque metadata:

```bash
deyta memory remember "Ticket resolved by restarting the worker" \
  --external-id ticket-0001 \
  --metadata-json '{"team": "support", "priority": "high"}'
```

- `--external-id` — your own identifier for the note. Reusing an identifier you've
  sent before re-remembers that note, replacing the matching document in place
  rather than appending a new one. Must be non-empty, ≤512 chars, and contain no
  whitespace (leading, trailing, or internal) nor any control, zero-width, or
  Unicode format code point — the value must be printable. Validated client-side,
  so a bad value fails fast without contacting the daemon.
- `--metadata-json` — an inline JSON **object**; its contents are opaque and kept
  verbatim. Anything that isn't a valid JSON object (invalid JSON, or an array/scalar)
  is rejected before any request is made.

Both flags are optional and default to unset. These are the single-note equivalents of
the `external_id` and `metadata` fields in the [JSON document format](#json-document-format)
used by `deyta ingest`.

## Backends

- **embedded** (`sqlite_lance`) — SQLite + LanceDB, fully in-process, zero infra. Default for quickstart.
- **postgres** — Postgres + pgvector + Neo4j via `deyta db up` (vendored Docker Compose,
  pinned to `pgvector/pgvector:pg17` and `neo4j:2025.12.1`).

Upgrading to Khora 0.26.0 (title keyword search): an **embedded** database created by an
earlier version keeps its original keyword index, so recall will not match on `title`
alone — there is no in-place upgrade for it, and nothing reports this at the default
settings. Run `deyta init` against a fresh database and re-ingest to enable it. Existing
content search is unaffected either way. On **postgres** no recreation is needed: the
server migrates in place on its first start after the upgrade, and documents already
ingested become title-searchable once it finishes. That migration rewrites the whole
chunk table and blocks reads on it while it runs, so expect that first start to take
substantially longer than a normal restart on a large database.

## Configuration & state

- `~/.config/deyta/deyta.toml` — main config: backend, server port, LLM settings,
  default ontology, namespace aliases, active namespace.
- `~/.config/deyta/.env` — secrets (`DEYTA_OPENAI_API_KEY`; optionally
  `DEYTA_SERVER_API_KEY` to pin the local server's key). Note: a `DEYTA_API_KEY` line
  here is treated as the CLIENT token and overrides the active context's stored key —
  leftover legacy lines cause 401s against a freshly keyed server; remove or rename
  them (see Migration).
- `~/.config/deyta/config.toml` — contexts (local/cloud). `auth.json` holds cloud tokens.
- `~/.config/deyta/daemon.json` — runtime state for a detached daemon (pid/port).

`deyta config` opens an interactive editor for `deyta.toml` — arrow keys to navigate,
Enter to edit a value inline, `s` to save, `q` to quit. The editor has two views:

- **Simple** (default) — shows backend, server, LLM, ontology, and secrets, plus any
  Khora overrides you've already set.
- **Advanced** (press `a`) — shows every available Khora tuning parameter (~290 across
  8 sections) with their defaults. Values you haven't changed appear dimmed. Edit any
  parameter inline; clear a value to reset it to the default.

For scripting: `deyta config get llm.model` / `deyta config set llm.model gpt-4o`.

Ontology: `deyta init` writes a generic default `entity_types` / `relationship_types`
so `deyta ingest run` works with no flags; override per run with `--entity-types` /
`--relationship-types`.

## Bulk ingest

`deyta ingest` is a command **group** with two subcommands:

- `deyta ingest run <path> [<path>...]` — walk the paths, chunk + remember, streaming
  each sub-batch to the daemon.
- `deyta ingest status --log <path> [<path>...]` — summarize one or more progress logs
  offline (see [`ingest status`](#ingest-status) below).

`deyta memory ingest` mounts the **same** group, so `deyta memory ingest run …` /
`deyta memory ingest status …` behave identically.

> The old single-form `deyta ingest <path>` was removed. Running it now prints a
> migration error (`deyta ingest <path>` moved — use `deyta ingest run <path>` instead.)
> and exits non-zero. Use `deyta ingest run <path>`.

### `ingest run` flags

| Flag | Default | Purpose |
|---|---|---|
| `--batch-size N` | `25` | Documents per sub-batch. Must be `>= 1` (a smaller value errors before any work). |
| `--log <path>` | timestamped file in the cwd | Append-only JSONL progress log (see below). Enables resumable runs. |
| `--continue` | off | Resume mode: process only documents **never attempted** in `--log`. |
| `--failed-only` | off | Resume mode: re-attempt only documents whose **last** logged status is `failed`. |

Run modes:

- **Fresh** (no `--log`, or a `--log` that doesn't exist / is empty) — process the whole
  corpus. Without `--log`, a timestamped log is written to the current directory anyway.
- **`--continue`** — process only never-attempted documents (new files added since the
  last run, or documents a crashed run never reached).
- **`--failed-only`** — re-attempt only documents whose last recorded status is `failed`.

`--continue` and `--failed-only` are **mutually exclusive**, and each requires an
explicit `--log` that already has recorded runs. Re-running against an existing `--log`
**without** a mode flag is an error with guidance to pick `--continue`, `--failed-only`,
or a new `--log`.

### Sub-batching & retry

`ingest run` splits the corpus into sub-batches of `--batch-size` and sends them
**sequentially**. Batch assembly is deterministic and preserves first-seen order;
documents sharing an `external_id` are **never** placed in the same sub-batch (a
collision is deferred to a later batch).

> **Idempotent replay requires `external_id`.** A batch is only safe to replay when
> **every** document in it carries a stable `external_id` — the server upserts on that
> id, so re-sending an unchanged document is deduped. Documents **without** an
> `external_id` have no dedup key, so if a partially-failed batch that contains id-less
> documents is replayed (via `--failed-only`), the already-ingested id-less documents can
> be **duplicated**. Give bulk documents an `external_id` when you need safe re-runs.

Each sub-batch is streamed with retry:

- **Retryable** failures — HTTP `429`, `500`, `502`, `503`, `504`, and mid-stream drops
  (truncated SSE frame, dropped connection, or a stream that sends **no bytes for 90s**)
  — are retried with **exponential backoff** (base **5s**, factor **2**, max **3
  attempts**: waits of 5s then 10s). After the attempts are exhausted, every document in
  the batch is marked `failed` and the run continues to the next batch.
- A batch is auto-retried **only if every document in it has an `external_id`** (so the
  replay is idempotent). An id-less batch that fails is marked `failed` immediately with
  the reason `not retried: documents lack external_id`.
- **Non-retryable** statuses — `401`, `403`, `404`, `422`, and any other 4xx — **abort
  the run immediately**.
- **Circuit breaker** — if the **first** sub-batch lands nothing (fails entirely), the
  run aborts rather than grinding through a mis-configured corpus. The abort message
  leads with the underlying error when there is one — a locked local store prints
  `database is locked`, not a guess — and falls back to the generic hint (check server
  config, model, and API keys) only when no detail is available.

> **Stalled streams.** Extraction can be silent for minutes on a large chunk, so the
> server emits an SSE keepalive comment every **15s** while a batch is running. 90s of
> total byte silence therefore means the server is gone, not slow — which is why the
> client gives up and retries at that point instead of hanging forever. Any proxy,
> load balancer, or VPN gateway between the CLI and the server needs an **idle timeout
> above ~30s** and **unbuffered (streaming) proxying** for this to work; the response
> also carries `Cache-Control: no-cache` and `X-Accel-Buffering: no`, though the latter
> is only honoured by nginx.
>
> **Talking to an older server.** The 90s budget assumes the server sends those
> keepalives. A server older than this feature stays silent for the whole extraction,
> so a slow batch can trip the budget and be retried (or failed) for no reason. Set
> `DEYTA_STREAM_READ_TIMEOUT` to raise the ceiling, or to **`0`** to wait indefinitely
> as before. Upgrading the server is the real fix — this is the escape hatch for the
> window where the CLI is newer than the daemon it targets.

### Progress log (JSONL)

With `--log` (or the auto-named default) the run appends one JSON object per line. An
**advisory file lock** is held for the whole run, so two runs can never share a log. The
log is opened **before any request** — an unwritable log path exits immediately, before
a single document is sent. Lines are **flushed (and fsync'd) per batch**, so a crashed
run still leaves a resumable log. Each document is keyed by its `external_id`, falling
back to its `source` when it has none.

Line types and fields:

- `run_started` — `{ "type": "run_started", "run_id", "started_at", "mode",
  "namespace_id", "paths", "batch_size" }`
- `doc` — `{ "type": "doc", "run_id", "batch", "external_id", "source", "doc_seq",
  "origin", "status", "skip_reason", "error", "ts" }` (one per document; `status` is
  `ingested` or `failed`). For id-less documents the resume identity is `origin` (the
  physical file the document was read from) plus `doc_seq` (its 0-based index within
  that file), so id-less documents in different files never share a key even when they
  declare the same caller-supplied `source`. `origin` is `null` for `external_id`-keyed
  documents (which key on the id alone).
- `run_completed` — `{ "type": "run_completed", "run_id", "completed_at",
  "totals": { "ingested", "replaced", "skipped", "failed" } }`

> **Attribution granularity.** The batch endpoint returns an aggregate result per
> sub-batch (totals for processed / skipped / failed), not a per-document outcome. The
> progress log therefore records outcomes at batch granularity: a sub-batch that completes
> with **zero reported failures** logs every document `ingested`; a sub-batch that errors,
> exhausts retries, **or reports any failures in its aggregate result** logs every document
> `failed`. Because the endpoint does not say *which* documents failed, a partially-failed
> sub-batch is conservatively marked failed in full — this keeps the genuinely-failed
> documents recoverable via `--failed-only`. Replaying a batch is cheap and safe **for
> documents with an `external_id`** (re-ingesting an unchanged document is deduped
> server-side); id-less documents that already succeeded in that batch have no dedup key
> and can be **duplicated** on replay. A consequence of the conservative marking is that
> `failed` totals can over-count for a partially-failed batch. Server-side
> `skipped`/replaced counts within a
> successful sub-batch are likewise not attributed to individual documents. The
> `run_completed` totals are the sum of these per-document statuses, and
> `--continue`/`--failed-only` operate on this batch-granular record. Switching to a
> per-document durable submit path is deliberately out of scope for this change.

### `ingest status`

`deyta ingest status --log <path> [<path>...]` summarizes finished logs **offline** — it
never constructs a client and never contacts the daemon. It prints cumulative totals by
outcome, the list of failed documents, and a per-run history (mode, start time, duration,
per-run totals). When input paths are also supplied it reconciles them against the log(s)
and prints `remaining: N unattempted`. Corrupt or truncated log lines are tolerated —
counted and noted, never fatal.

```bash
deyta ingest status --log deyta-ingest-20260115-101500.jsonl ./docs
```

### Live progress

On a terminal, `ingest run` shows a single **corpus-level** Rich progress bar that only
moves forward — sub-batches advance the one bar (they never reset it), so a mid-batch
retry that rewinds the server's `processed` count can't double-count.

Under a pipe, `nohup`, or CI (no terminal), the live bar is replaced by **one timestamped
line per completed batch** so `tail -f` shows progress:

```
[HH:MM:SS] batch k/K done — cumulative X/Y docs, f failed
```

### JSON document format

`deyta ingest run` accepts one or more files/directories. A `.json` file (case-insensitive)
is parsed as a document file — either one document (a single JSON object) or many (an
array of objects). Every other extension is ingested as raw text, so mixed `.json` + `.md`
trees work in a single run:

```bash
deyta ingest run ./notes ./exports/tickets.json ./readme.md
```

Each JSON document is a single object. Only `content` is required:

```json
{
  "content": "Full text of the document. Required, must be a non-empty string.",
  "title": "Optional title. Indexed for keyword search, not just displayed.",
  "source": "tickets/ticket-0001.md",
  "source_type": "markdown",
  "source_name": "support-export",
  "source_url": "https://example.com/tickets/1",
  "source_timestamp": "2026-01-15T10:30:00Z",
  "external_id": "ticket-0001",
  "metadata": { "team": "support", "priority": "high" }
}
```

Fields:

| Field | Required | Notes |
|---|---|---|
| `content` | yes | Non-empty string. |
| `title` | no | Defaults to `""`. Indexed for keyword search alongside `content`, so recall can match a document by its title alone. On a database predating Khora 0.26.0 this needs the upgrade to have taken effect — automatic on postgres once the migration finishes, but embedded databases have to be recreated (see Backends). |
| `source` | no | Defaults to the file's path relative to the walk root. |
| `source_type`, `source_name`, `source_url` | no | Strings; passed through untouched. |
| `source_timestamp` | no | ISO-8601 string (see below). |
| `external_id` | no | Non-empty, ≤512 chars, no whitespace. |
| `metadata` | no | A JSON object; contents are opaque and kept verbatim. |

- **Provenance:** `source` alone gets a **client-side** default — the file's path relative to
  the walk root (see the table). The other provenance fields (`source_type`, `source_name`,
  `source_url`, `source_timestamp`) are passed through untouched; when one of *those* is
  omitted, the **server** fills its own default.
- **`source_timestamp`** is an ISO-8601 string — e.g. `2026-01-15T10:30:00Z`,
  `2026-01-15T10:30:00+00:00`, or date-only `2026-01-15` (explicit offsets are allowed).
  It is validated with the same coercer the server applies, which also accepts naive and
  second-less `T`-forms; these examples are illustrative, not the full set.
- **Leniency:** unknown top-level keys are kept, never an error. To omit an optional field,
  leave the key out — do **not** set it to `null` (`null` is the wrong type, hence invalid).
- **Validation:** a wrong-typed known field, a missing/empty `content`, an `external_id`
  that is empty / longer than 512 chars / contains whitespace, or an unparseable
  `source_timestamp` makes that **one** document invalid — it is reported to stderr and
  skipped, and the run continues. The remaining documents are still ingested, but the run
  ends with exit `1` (see **Exit codes**).
- **Walk order** is deterministic across runs: input paths in command-line order, files
  within a path in alphabetical (byte) order, documents within an array in array order.
- **Duplicate `external_id`s** across the corpus are allowed; a summary line reports how
  many distinct ids appear more than once. Same-request ordering with duplicate ids is not
  guaranteed.
- **Exit codes** turn on documents *failed*, not documents *discovered*. `0` when the run
  completes with nothing in the `failed` total. `1` when **any** document fails — including
  one that fails only validation, since validation failures count toward that total — and
  likewise when **no documents are discovered at all** (a nonexistent input path, or a corpus
  with no parseable units, such as an empty directory or an empty `[]` array), when a batch
  aborts on a non-retryable server error, or on a preflight error such as `--batch-size` below
  `1` or an unusable `--log`. `2` is reserved for command-line misuse — the legacy
  `deyta ingest <path>` form, or a missing required argument. `--dry-run` always exits `0`:
  validation failures are still reported to stderr, but nothing is sent and nothing fails.
- **Extension dispatch (known limitation):** only files ending in `.json` are parsed as
  document files; every other extension — including `.jsonl` (JSON Lines) — is ingested as
  raw text, so a `.jsonl` file becomes a single text document, not one document per line.
  There is currently no way to force a `.json` file to be ingested as raw text. Incidental
  `.json` files that aren't document files (e.g. `package.json`, `tsconfig.json`) are
  reported as validation failures (missing `content`) and skipped — keep them out of the
  ingest path.

### Custom extraction expertise (`--expertise-file`)

Flat entity/relationship name lists are a floor, not a ceiling. For a specialized
corpus you can supply a full **expertise config** — a custom extraction prompt,
a system prompt, and typed entity/relationship definitions *with descriptions* —
in Khora's native format:

```bash
deyta ingest run ./corpus --expertise-file ./oncology.yaml
```

The file is a YAML or JSON document in Khora's `ExpertiseConfig` format. deyta-cli
defines no schema of its own here — see Khora's expertise-configuration
documentation for the authoritative field reference. The file is loaded and
validated client-side (via Khora's own loader) **before any request**, so a bad
file fails fast with Khora's error instead of surfacing mid-ingest.

- **Name derivation:** when `--expertise-file` is given, the request's
  `entity_types` / `relationship_types` name lists are derived from the config's
  typed definitions. An explicit `--entity-types` / `--relationship-types` flag
  still overrides its side; the expertise config rides along regardless.
- **Server default (`DEYTA_EXPERTISE`):** a server/container can set a default
  expertise for every request. The value is either a path to a YAML/JSON file or
  an inline JSON object (a leading `{` selects the inline form); it is parsed at
  startup, so a malformed value fails the server fast. This default and the
  precedence chain below apply to the HTTP remember and ingest routes only; the
  MCP `remember` tool uses khora's own defaults and is not affected by
  `DEYTA_EXPERTISE`.
- **Precedence:** a per-request expertise (from `--expertise-file`) overrides the
  `DEYTA_EXPERTISE` default, which overrides none. `DEYTA_ENTITY_TYPES` /
  `DEYTA_RELATIONSHIP_TYPES` remain the server's name-list defaults.

### Advanced Khora settings (`[khora.*]`)

Beyond the basics above, every Khora tuning parameter can be set in `deyta.toml`
under `[khora.<section>]` tables that mirror Khora's own config sections:

| Section | What it tunes |
|---|---|
| `[khora.recall_vectorcypher]` | Recall engine: fusion weights, graph traversal depth, BM25 channel, cross-encoder + LLM reranking, extraction concurrency |
| `[khora.llm]` | Temperature, max_tokens, retries, concurrency, extraction model, connection pool |
| `[khora.pipeline]` | Chunking strategy/size/overlap, conversation grouping, selective entity extraction |
| `[khora.query]` | Query pipeline: channel weights, entity linking, HyDE, multi-stage limits, temporal resolver |
| `[khora.storage]` | Pool sizes, HNSW index parameters (connection URLs/credentials are managed by deyta and rejected here) |
| `[khora.hooks]`, `[khora.tenancy]`, `[khora.dream]` | Semantic hooks, tenancy mode, dream-phase maintenance |

Example:

```toml
[khora.recall_vectorcypher]
enable_reranking = true
enable_llm_reranking = true
llm_reranking_mode = "always"
fusion_vector_weight = 0.6
bm25_top_k = 50

[khora.llm]
temperature = 0.7
max_concurrent_llm_calls = 10

[khora.pipeline]
chunking_strategy = "semantic"
chunk_size = 512
```

Keys are validated against the installed Khora version's config classes — a typo
fails with a suggestion instead of being silently ignored. Scripting:
`deyta config set khora.recall_vectorcypher.bm25_top_k 40` (values are parsed as
JSON: `true`, `0.4`, `[1, 2]`).

## Deploy to Fly.io

`deyta deploy fly` runs Postgres + Neo4j + the Deyta daemon on a single Fly
Machine with one persistent volume mounted at `/data` (both databases store
their data there — it survives restarts, redeploys, and resizes).

- **Multiple deployments** — pass `--name` to create a new deployment even when
  one already exists. Each deployment gets its own context. Without `--name`,
  the CLI detects the existing deployment and offers to redeploy it.

  ```bash
  deyta deploy fly                        # creates deyta-quiet-maple (random name)
  deyta deploy fly --name khora-staging   # creates a second deployment
  ```

- **Machine sizing** — defaults to `performance-4x` / 16 GB with a 20 GB volume.
  Override at deploy time (`--vm-size performance-2x --vm-memory 8gb
  --volume-size 40`) or later with `deyta deploy scale` (in-place update, no
  image rebuild; volumes can grow, never shrink). A plain redeploy reuses the
  deployment's stored sizing. The entrypoint gives Neo4j a quarter of machine
  memory for JVM heap and a quarter for page cache.
- **Config of record** — each deployment keeps a config snapshot under
  `~/.config/deyta/deploy/<app>/deyta.toml`. With the Fly context active,
  `deyta config` edits that snapshot (not your local `deyta.toml`) and offers
  to apply it. `deyta deploy config` pushes it to the running app as Fly
  secrets — no image rebuild. Redeploys ask which config to use (deployment's
  current config, local `deyta.toml`, or step-by-step; `--config-source
  deployed|local` for scripts) and never silently pick up local settings.
- **Restarts and downtime** — applying config or scaling restarts the machine
  behind Fly's health checks. There is no zero-downtime path with this
  architecture: a Fly volume attaches to exactly one machine and both databases
  live on it, so a second machine can't take over the data. Config-only applies
  skip the image pull; the restart window is dominated by Neo4j startup
  (roughly 20–60 s), during which Fly's proxy queues incoming requests. One
  exception: the first restart after a Khora upgrade also applies pending
  database migrations before the server accepts traffic, and that can take far
  longer than 20–60 s on a large database. `deyta deploy config` and `deyta
  deploy scale` stop waiting after 120 s and report the deploy as unhealthy —
  if that happens during a migration, check `flyctl logs` and let it finish
  rather than retrying, since restarting the machine rolls the migration back
  and it starts over.

### Sharing a deployment with teammates

The person who runs `deyta deploy fly` gets a local context wired up
automatically. A teammate who needs to use the same deployment can connect
with `deyta context add`:

```bash
deyta context add deyta-quiet-maple \
    --host https://deyta-quiet-maple.fly.dev \
    --api-key <key>
```

This creates a local context pointing at the existing deployment and switches
to it. All `deyta` commands (`query`, `ingest`, `config`, etc.) now target that
deployment. The API key to pass is the deployment's server key — `deyta deploy
fly` never prints it, but stores it (mode `0600`) in `~/.config/deyta/auth.json`
under the deployment's context name. Read it from there and share it through a
secure channel. If that file is unavailable, rotate the key with `deyta deploy
fly` — it sets the new `DEYTA_SERVER_API_KEY` Fly secret **and** updates the
deployment's context token in `auth.json` in one step. (A manual `fly secrets set
DEYTA_SERVER_API_KEY=…` changes only the server side; you must then update that
context's client token in `auth.json` to match, or clients will get `401`.) Fly
never reveals a secret's value after it is set.

To switch back to local development:

```bash
deyta context use local
```

To list all contexts or remove one:

```bash
deyta context list
deyta context remove deyta-quiet-maple   # does not destroy the deployment
```

## Authentication

Auth is **always on**, and the two sides use **separate** environment variables
so a client credential is never mistaken for the server's secret:

| Variable | Role | Who reads it |
|---|---|---|
| `DEYTA_SERVER_API_KEY` | The secret the **server** requires on every request. | The server process only. |
| `DEYTA_API_KEY` | The bearer token a **client** sends. | The CLI / SDK when talking to a server. |

The server treats `DEYTA_SERVER_API_KEY` as mandatory: it **refuses to start**
without it (a raw `deyta-server`, or a container launched without the variable,
fails fast with a clear message) and then requires `Authorization: Bearer <key>`
on every request. There is no "unauthenticated" mode. Clients send their key via
`DEYTA_API_KEY`; the TypeScript SDK sends the `apiKey` you pass at construction.

### Where the key comes from (CLI-side generation)

You rarely type a server key by hand — the CLI generates one for you and wires
both sides up:

- **Local launcher** (`deyta serve` / `deyta up`): if no server key exists, the
  CLI generates one, writes it to a `0600` key file under `~/.config/deyta/`, and
  starts the daemon with it exported as `DEYTA_SERVER_API_KEY`. The matching
  client token is stored so local `deyta` commands authenticate automatically.
- **Deploy** (`deyta deploy fly`): the CLI generates a fresh server key, sets it
  as the Fly secret `DEYTA_SERVER_API_KEY`, and saves the client-side token in
  `~/.config/deyta/auth.json` (mode `0600`) under the deployment's context.

### Retrieving a key (deliberate read only)

Generated keys are **never printed to the terminal or logs** — deploy and launcher
output name only the storage location, never the value (masked or otherwise). When
you genuinely need the raw value (to add another client or share with a teammate),
read it deliberately from where it is stored:

- **Local:** the `0600` key file under `~/.config/deyta/` (launcher), or the
  client token in `~/.config/deyta/auth.json`.
- **Fly deployment:** `~/.config/deyta/auth.json` on the deploying machine is the
  source of truth — Fly itself never exposes a secret's value after it is set. If
  `auth.json` is unavailable, rotate the key with `deyta deploy fly`: it updates
  both the Fly secret and the deployment's context token in `auth.json`. A manual
  `fly secrets set` changes only the server credential, so the context's client
  token in `auth.json` must be updated to match, or clients will get `401`.

Do not scrape keys out of logs — they aren't there by design.

### Migrating an existing Fly deployment

Older deployments stored the server secret under `DEYTA_API_KEY`. Because the
server now reads `DEYTA_SERVER_API_KEY` and refuses to start without it, set the
new secret **first**, confirm the app is healthy, and only then remove the legacy
one:

```bash
fly secrets set DEYTA_SERVER_API_KEY=<same token>   # reuse the same value so existing clients keep working
# …verify the app is healthy…
fly secrets unset DEYTA_API_KEY
```

Re-running `deyta deploy fly` also provisions the new secret for you.

### Headless usage (no config files, no prompts)

Every command runs fully non-interactively when you supply the host and token via
the environment. No `~/.config/deyta`, no `deyta.toml`, and nothing read from
stdin. With no local config there is no active namespace either, so memory
commands need an explicit `--namespace` (`-n`):

```bash
DEYTA_HOST=https://deyta-quiet-maple.fly.dev \
DEYTA_API_KEY=<client-token> \
    deyta query "your question" --namespace my-ns
```

`DEYTA_HOST` selects the target server (highest-precedence, above any active
context), `DEYTA_API_KEY` is the bearer token sent with the request, and
`--namespace` names the namespace to operate on — ideal for CI, agents, and
one-off scripts against a remote deployment.

## Releasing

The package builds with hatchling; the `deyta` command comes from the
`[project.scripts]` entry point in `pyproject.toml`. Pushing a `vX.Y.Z` tag
publishes to PyPI (via GitHub Actions Trusted Publishing) and bumps the Homebrew
tap. See [RELEASING.md](RELEASING.md) for the one-time setup and the release steps.
