Metadata-Version: 2.4
Name: mostlyright-data
Version: 0.9.0
Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
Requires-Python: <3.14,>=3.13
Provides-Extra: build
Requires-Dist: hatchling==1.28.0; extra == 'build'
Requires-Dist: uv-build==0.9.26; extra == 'build'
Provides-Extra: dev
Requires-Dist: cmarkgfm>=2025.10.22; extra == 'dev'
Requires-Dist: croniter==6.2.4; extra == 'dev'
Requires-Dist: cryptography<51,>=50.0.0; extra == 'dev'
Requires-Dist: et-xmlfile==2.0.0; extra == 'dev'
Requires-Dist: idna<4,>=3.10; extra == 'dev'
Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'dev'
Requires-Dist: numpy==2.5.1; extra == 'dev'
Requires-Dist: openpyxl==3.1.5; extra == 'dev'
Requires-Dist: packaging<27,>=24; extra == 'dev'
Requires-Dist: pandas==3.0.3; extra == 'dev'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'dev'
Requires-Dist: polars==1.42.1; extra == 'dev'
Requires-Dist: pyarrow==23.0.1; extra == 'dev'
Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: referencing<1,>=0.35; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Provides-Extra: hosted
Requires-Dist: mostlyright-studio-client-v3==3.0.0; extra == 'hosted'
Provides-Extra: local
Requires-Dist: croniter==6.2.4; extra == 'local'
Requires-Dist: cryptography<51,>=50.0.0; extra == 'local'
Requires-Dist: et-xmlfile==2.0.0; extra == 'local'
Requires-Dist: idna<4,>=3.10; extra == 'local'
Requires-Dist: numpy==2.5.1; extra == 'local'
Requires-Dist: openpyxl==3.1.5; extra == 'local'
Requires-Dist: packaging<27,>=24; extra == 'local'
Requires-Dist: pandas==3.0.3; extra == 'local'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'local'
Requires-Dist: polars==1.42.1; extra == 'local'
Requires-Dist: pyarrow==23.0.1; extra == 'local'
Provides-Extra: local-search-neural
Requires-Dist: anyio==4.14.2; extra == 'local-search-neural'
Requires-Dist: certifi==2026.7.22; extra == 'local-search-neural'
Requires-Dist: click==8.4.2; extra == 'local-search-neural'
Requires-Dist: filelock==3.32.2; extra == 'local-search-neural'
Requires-Dist: flatbuffers==25.12.19; extra == 'local-search-neural'
Requires-Dist: fsspec==2026.7.0; extra == 'local-search-neural'
Requires-Dist: h11==0.16.0; extra == 'local-search-neural'
Requires-Dist: hf-xet==1.6.0; extra == 'local-search-neural'
Requires-Dist: httpcore==1.0.9; extra == 'local-search-neural'
Requires-Dist: httpx==0.28.1; extra == 'local-search-neural'
Requires-Dist: huggingface-hub==1.27.0; extra == 'local-search-neural'
Requires-Dist: idna==3.18; extra == 'local-search-neural'
Requires-Dist: numpy==2.5.1; extra == 'local-search-neural'
Requires-Dist: onnxruntime==1.28.0; extra == 'local-search-neural'
Requires-Dist: packaging==26.3; extra == 'local-search-neural'
Requires-Dist: protobuf==7.35.1; extra == 'local-search-neural'
Requires-Dist: pyyaml==6.0.3; extra == 'local-search-neural'
Requires-Dist: tokenizers==0.23.1; extra == 'local-search-neural'
Requires-Dist: tqdm==4.70.0; extra == 'local-search-neural'
Requires-Dist: typing-extensions==4.16.0; extra == 'local-search-neural'
Provides-Extra: release
Requires-Dist: croniter==6.2.4; extra == 'release'
Requires-Dist: cryptography<51,>=50.0.0; extra == 'release'
Requires-Dist: et-xmlfile==2.0.0; extra == 'release'
Requires-Dist: idna<4,>=3.10; extra == 'release'
Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'release'
Requires-Dist: numpy==2.5.1; extra == 'release'
Requires-Dist: openpyxl==3.1.5; extra == 'release'
Requires-Dist: packaging<27,>=24; extra == 'release'
Requires-Dist: pandas==3.0.3; extra == 'release'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'release'
Requires-Dist: polars==1.42.1; extra == 'release'
Requires-Dist: pyarrow==23.0.1; extra == 'release'
Provides-Extra: viewer
Requires-Dist: nbconvert==7.17.1; extra == 'viewer'
Requires-Dist: nbformat==5.10.4; extra == 'viewer'
Description-Content-Type: text/markdown

# mostlyright-data

`mostlyright-data` converts a question and selected sources into a reproducible Parquet
dataset. An external agent may select sources and author a declarative recipe. A human approves the
exact recipe digest. Deterministic workers acquire bytes, transform data, run checks, and seal an
immutable Build.

For question-to-dataset work, the external host agent owns orchestration through delivery of a
verified Build and its table notebook. Exact proposed-contract or Recipe-digest approval is the
only user pause: typed refusals and failed checks drive bounded retry, supported substitution, or
a replacement Recipe. Workers remain fail-closed, changed Recipe digests require new approval,
and Studio alone owns release. If every registered lawful alternative is unsupported, the agent
reports that typed impossibility instead of bypassing a refusal or fabricating a dataset.

Source discovery is a host-owned, multi-lane search. `network_search_bundled: false` is a truthful
capability disclosure: the credential-isolated Harness does not contain or execute a browser. It is
not a switch that disables the host's public-web tools. For question-to-dataset work, the host must
search every available lane—registered Harness acquisition adapters, a configured sealed catalog,
public web search, official provider catalogs or APIs, and user-supplied sources—with no preferred
lane. It then merges and deduplicates source options, inspects them through registered boundaries, and
compares them on the canonical `harness-source-fitness.v1` dimensions. Every lane cites an immutable
host capability receipt, every query cites a host-tool receipt, and every comparison cites source
or receipt evidence. A host receipt is a strict JSON object whose SHA-256 is calculated over its
canonical UTF-8 JSON bytes; preserve those bytes in `research.ipynb` or record the exact durable
host-result reference beside the digest. Those receipts and the report remain untrusted host input: `mr-data
source-audit` checks their structure and internal bindings, but does not claim a browser ran. It
still performs no search, fetch, approval, or acquisition itself.

Selected and eligible sources also record the exact promoted inspection coordinate—locator kind,
credential-free locator, certified Reader identity plus decode-options digest (or `null` for native
CSV handling), format, and host platform—and its immutable inspection-receipt digest. Typed
acquisition refusals are not fitness failures. Non-final retries remain rejected; recovery and
lawful exhaustion are terminal, and an unsupported decision requires every typed-refusal route to
be lawfully exhausted.

The repository includes a complete report example and the canonical host-receipt objects whose
digests it cites. Audit the report with:

```bash
uv run --extra dev mr-data source-audit \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --discovery-report examples/city-temperature/source-discovery-report.json \
  --json
```

The package version is `0.9.0` and requires Python `>=3.13,<3.14`.

## Two install profiles

`mr-data` is one binary with two install profiles, and the command names are the same in both.

| Profile | Install | Where it runs | What it is |
| --- | --- | --- | --- |
| Hosted (the default) | `pip install mostlyright-data` | Any operating system with CPython 3.13 -- Linux, macOS and Windows alike | Pure Python. One `py3-none-any` wheel, no compiled extension in the install closure and, today, no third-party package at all. It signs in, submits work to the Mostly Right backend, streams a run's live progress, downloads the dataset it produced, and shows the receipts. |
| Local | `pip install "mostlyright-data[local]"` | Linux; installable on macOS, untested there | Everything above plus the engines, the certified Readers, the Clean room, and every build command below. Not a product surface: ADR 0021 makes every lane hosted, and this extra is operator and developer tooling for the machines that build the workers. |

One wheel serves every platform, and that is checked rather than asserted: `thin-client-smoke`
builds exactly one artifact and installs **that same file** on ubuntu, macOS and Windows before
running the hosted command surface on each.

Windows gets the hosted profile and not the local one, and the reason is not that nobody has
ported it: the local lane runs every acquisition and every Reader behind a kernel boundary built
from POSIX process control -- `fork` and cgroups v2 -- and Windows has neither. Since ADR 0021
that is also true of macOS: Seatbelt was the second boundary implementation, it existed so a
person's Mac could run a lane, and it left with the product surface that needed it. Note that
`pip install "mostlyright-data[local]"` *succeeds* there, because
the engines all publish Windows wheels, so what you installed is not what decides it. `mr-data`
asks the machine, and every local command refuses **by name** with that reason rather than
running into a traceback later. One consequence is recorded rather than hidden: the credential
`mr-data login` stores is written at mode `0600` on Linux and macOS, and on Windows is protected
by the ACL your user profile directory already carries, which is a weaker statement.

`mr-data` chooses the lane by what is installed. With `[local]` present every invocation behaves
exactly as it always has; `--hosted` on `build`, `run`, `watch`, `status` or `deploy-dataset`
selects the backend instead. Without `[local]`, the hosted lane is the only lane, and a command
that runs on this machine refuses **by name** and says what to install rather than failing on a
missing import.

Where a hosted command runs is not a variance a person has to track: a submission prints the
dashboard address for the run it queued, and `mr-data watch <run-id>` follows that run's event
log live, resuming from its last cursor when the connection is cut rather than replaying it.

## Supported platforms

| Platform | Installed capability |
| --- | --- |
| Anything CPython 3.13 runs on -- Linux, macOS, Windows | The hosted profile, complete: `login`, `whoami`, and every hosted command. This is the product, and it is the row that matters. |
| Ubuntu 24.04 x86_64, CPython 3.13.14 | Additionally the `[local]` extra: every certified Reader over local files and public HTTPS, including bounded archives, workbooks, and optional GRIB. The Clean room uses a private network namespace and an operator-bounded cgroup root. This is the platform the hosted workers are, which is why it is the one that is tested. |
| macOS 14 arm64 | The hosted profile, and `[local]` installs. Nothing under `[local]` is tested there any more: ADR 0021 retired `parity-macos`, `sandbox-macos`, and the macOS legs of the `local-*` workflows along with the local lane. Clean-room execution refuses -- `SANDBOX_MEMORY_BOUNDARY` for a Reader, `SANDBOX_OS_BOUNDARY` for anything else -- because the Seatbelt boundary that used to admit it is gone. A logged-in single-address credential-free public HTTPS acquisition still runs in Studio's reviewed Linux/amd64 hosted Clean room. Bounded collections require both local attestations, so they need a host that can produce them. Catalogue authoring, which is local by design, still works. |
| Windows | The hosted profile. `pip install "mostlyright-data[local]"` succeeds, because the engines all publish Windows wheels, so what you installed is not what decides it: every local command refuses **by name** and says why. |

The GRIB Reader is optional. If a compatible `mostlyright-grib` binding is unavailable, first use
fails with `READER_DEPENDENCY_UNAVAILABLE`. It accepts one extracted GRIB2 record and rejects a
multi-record GFS or HRRR file.

## Install from a checkout

Update the persistent uv tool and install the complete matching skill for both Codex and Claude
Code with one command from the checkout:

```bash
uv run --no-project --python 3.13 python scripts/install_checkout.py
```

The installer resolves the checkout from its own path, reinstalls `mr-data` from that exact
directory and installs the exact pinned generated-client wheel alongside it, copies the complete `skills/mr-data-build` tree to both
personal skill directories, and then verifies `mr-data --version`, exact source-to-installed
package bytes, and exact source-to-installed skill-tree parity. The separately owned generated
Studio client is also exposed as the published distribution's `hosted` extra. The base Harness wheel remains client-free for
credential-free Courier images. It also binds every installed launcher to the
`[project.scripts]` entry point it declares.

`PATH` is checked last, after the verified generation is committed, because `PATH` belongs to your
shell rather than to the installation and an ambient shell problem must never restore the code the
install just replaced. If the uv tool bin directory is not on `PATH` at all, which is the ordinary
first-install case, the installer keeps the install, exits 0, and tells you to run
`uv tool update-shell` and open a new shell. If a different `mr-data` is found earlier on `PATH`,
the installer keeps the install and exits 2 with a warning naming both paths: typing `mr-data`
would otherwise still run the stale program. Put the uv tool bin directory earlier on `PATH`, or
remove the other one.

So read the exit code rather than testing it for zero: `0` and `2` both mean the new generation is
installed and verified, and only `1` means the install was refused and the prior generation
restored. Rerunning on `2` produces the same `2` until `PATH` changes.

It preflights both skill targets before changing the uv tool. If an installed skill differs, it
refuses with no mutation; inspect the named path and rerun with `--force` only when replacement is
intended. The installer retains private backups of the exact prior uv tool environment, all
Harness launchers, and both prior skill trees until final parity succeeds. Any failure restores
that whole prior generation, or removes every new artifact when no prior generation existed. A
single per-user lock serializes this entire operation with direct skill installs. Once exact final
parity is verified, the new generation is successful; failed best-effort backup cleanup emits a
warning and leaves the named private backup for manual recovery instead of undoing that generation.

For an isolated wheel smoke test instead of a persistent global tool update:

```bash
artifact_dir="$(mktemp -d)"
uv build --wheel --out-dir "$artifact_dir"
set -- "$artifact_dir"/mostlyright_data-*.whl
test "$#" -eq 1 && test -f "$1"
wheel="$1"
uv venv .work/harness-install --python 3.13
uv pip install --python .work/harness-install/bin/python "${wheel}[local]"
.work/harness-install/bin/mr-data --version
.work/harness-install/bin/mr-data inventory --json
```

### Run ordinary CSV with the installed binary

```bash
mkdir -p .work
.work/harness-install/bin/mr-data init .work/installed-city-temperature \
  --run-id installed-city-temperature \
  --input-root examples/city-temperature \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --plan examples/city-temperature/question-plan.json \
  --semantics examples/city-temperature/semantics.json \
  --json
.work/harness-install/bin/mr-data plan .work/installed-city-temperature --json
.work/harness-install/bin/mr-data run .work/installed-city-temperature --json
.work/harness-install/bin/mr-data verify .work/installed-city-temperature --json
```

The workspace must not exist before `init`. `run` seals the Build under `<workspace>/result` and
writes `table.ipynb` beside it.

Create a research directory, then start the local notebook viewer in a second terminal before
`init`. Keep the future workspace absent:

```bash
mkdir -p .work/installed-city-temperature-research
.work/harness-install/bin/mr-data view \
  --workspace .work/installed-city-temperature \
  --research-dir .work/installed-city-temperature-research \
  --json
```

The receipt names `url`, `research_notebook`, and `table_notebook`. Codex and Claude Code agents
always open `url` in their in-app browser and never pass `--open`; a host that is genuinely
terminal-only and is not Codex or Claude Code may use `--open`. For a direct Build, use
`mr-data view --run-dir PATH --research-dir RESEARCH_DIR`. `view` stays on loopback and starts
before source selection or Recipe approval. It streams atomic `research.ipynb` updates and adds
`table.ipynb` when the Build finishes. Keep an in-app viewer tab across approval turns. If a
successful Build reports an unavailable sidecar, run `mr-data notebook RUN_DIR --json`; the viewer
writes `table.ipynb` only when there is none and never replaces one that is there.
`mr-data watch` remains an opt-in operational diagnostic.

### Test the packaged Reader boundary

On Ubuntu 24.04 with cgroup v2, this command runs the installed Reader in the required memory and
network boundaries:

```bash
uv sync --locked --extra dev
sudo sh -eu -c '
  echo "+memory +pids" > /sys/fs/cgroup/cgroup.subtree_control
  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test
  echo 536870912 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
  echo 0 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.swap.max
  echo 64 > /sys/fs/cgroup/mostlyright-stock-reader-test/pids.max
  echo 1 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.depth
  echo 2 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.descendants
  echo "+memory +pids" > \
    /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.subtree_control
  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
'
sudo chown "$(id -u):$(id -g)" \
  /sys/fs/cgroup/mostlyright-stock-reader-test \
  /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.procs
test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
test ! -w /sys/fs/cgroup/cgroup.procs
runner_uid="$(id -u)"
runner_gid="$(id -g)"
sudo env \
  GITHUB_WORKSPACE="$PWD" \
  RUNNER_UID="$runner_uid" \
  RUNNER_GID="$runner_gid" \
  bash <<'STOCK_READER'
set -eu
cd "${GITHUB_WORKSPACE}"
echo "$$" > /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
exec 3</sys/fs/cgroup/mostlyright-stock-reader-test
exec 4</proc/self/ns/net
exec unshare --net -- setpriv \
  --reuid="${RUNNER_UID}" \
  --regid="${RUNNER_GID}" \
  --init-groups \
  .venv/bin/python -m pytest -q tests/test_stock_reader_workflow.py
STOCK_READER
find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
  -type d -name 'mostlyright-sandbox-*' -exec sudo rmdir {} +
test -z "$(find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
  -type d -name 'mostlyright-sandbox-*' -print -quit)"
sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test
test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test
```

For development, run the same workflow from the checkout:

```bash
uv sync --locked --extra dev
mkdir -p .work
uv run --extra dev mr-data init .work/city-temperature \
  --run-id golden-city-temperature \
  --input-root examples/city-temperature \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --plan examples/city-temperature/question-plan.json \
  --semantics examples/city-temperature/semantics.json
uv run --extra dev mr-data plan .work/city-temperature
uv run --extra dev mr-data run .work/city-temperature
uv run --extra dev mr-data verify .work/city-temperature
```

Add `--json` for one JSON object on success or failure. Failures exit nonzero. The command result is
the receipt; paths named by a receipt remain mutable after the command returns.

`--input-root` and `--output` may be relative paths. Relative paths are resolved from the current
working directory.

## Commands

| Command | Behavior |
| --- | --- |
| `mr-data catalog-search` | Search a sealed source catalog and return one entry with its query trace. |
| `mr-data catalog-fill` | Stage and publish one bounded credential-free CKAN, STAC, or SDMX catalog. |
| `mr-data catalog-harvest` | Stage a resumable Data.gov provider sweep, or verify its key boundary without making a request. |
| `mr-data catalog-author` | Convert one completed private harvest into bounded immutable authoring shards. |
| `mr-data catalog-publish` | Validate harvest and authoring receipts, then publish and independently read back one packed catalog generation. |
| `mr-data catalog-update` | Install one published catalog release after checking its exact archive digest, its store digest, and its descent from the generation already installed. |
| `mr-data index` | Rebuild the private local Build index under explicit discovery and byte limits. |
| `mr-data search` | Search exact public-catalog and private-Build snapshots under caller-supplied retrieval limits. |
| `mr-data source-audit` | Validate the host's multi-lane discovery report, source comparisons, proposals, and citations without searching or fetching. |
| `mr-data peek` | Report columns, types, and sample rows for a local file or public HTTPS resource, including an exact certified Reader selected with `--reader FAMILY@VERSION --reader-options JSON`. |
| `mr-data record` | Seal messages already recorded from a live source into bounded digest-named batches a build can read, writing down each dropped connection. It opens no connection and reaches no network. |
| `mr-data author` | Convert YAML into a Recipe. |
| `mr-data approve` | Bind approval to an exact Recipe digest. |
| `mr-data plan-check` | Execute a dataset plan in a temporary directory and report the result. |
| `mr-data show` | Inspect one Build. |
| `mr-data list` | List Builds under a root. |
| `mr-data diff` | Compare two Builds. |
| `mr-data review --local` | Run local checks without granting approval; exit 1 when a check fails. |
| `mr-data deploy-plan` | Validate a reviewed Build and write a deployment request without deploying it. |
| `mr-data deploy RUN_DIR` | Stage an exact verified Build, record and confirm its internal approval as the authenticated Editor, then activate and bind the Cloud Dataset. |
| `mr-data deploy-status RUN_DIR` | Report the state Studio holds for the run that deployment queued, optionally waiting for it with `--watch-seconds N`. It reads; it changes nothing Studio holds. |
| `mr-data deploy-dataset RUN_DIR` | Bring the Build that deployment's run released back into `RUN_DIR/hosted`, checked against the fingerprints the release sealed and checked again here, and write its dataset notebook. |
| `mr-data review-setup` | Generate review enrollment, keys, role tokens, and anchor input. |
| `mr-data notebook RUN_DIR` | Create or refresh `table.ipynb` for an existing Harness Build. |
| `mr-data view --workspace PATH` / `--run-dir PATH` | Serve the local notebook viewer, optionally pairing it with an explicit `--research-dir`. |
| `mr-data watch` | Serve the read-only event view. |
| `mr-data workflow` | Print the supported workflow stages and their authority boundaries. |
| `mr-data preflight` | Report build blockers for the host, for named options, or for a named workload measured against the engine's own ceilings. |
| `mr-data inventory` | Report installed capabilities. |
| `mr-data auth` | Validate the effective device credential with Cloud; manage metadata-only device revocation, remote-first logout, safe rotation, and explicit recovery; explain the ephemeral token boundary. |
| `mr-data login` | Sign in to the Mostly Right cloud with a device code and store a device credential. `login --force` is the compatibility alias for safe `auth rotate`, never an in-place truncate. |
| `mr-data whoami` | Compatibility alias for the remotely validated `mr-data auth status` answer. |

### The hosted lane

Twenty-five commands run on the Mostly Right backend. On a `[local]` install they take `--hosted`
to select that lane; on the hosted-only install they are what the command already does.
[Hosted parity](docs/HOSTED-PARITY.md) is the full ledger, generated from the classification the
software itself branches on; the table below is the submit-and-follow half of it, plus the
commands that write something other than a run.

| Command | Behavior |
| --- | --- |
| `mr-data build --hosted` / `mr-data run --hosted` | Queue one hosted initial build from an approved recipe proposal (`--recipe-proposal`, `--recipe-digest`, optionally `--resource-class standard\|large`) and print the dashboard address to watch it at. |
| `mr-data watch RUN_ID --hosted` | Stream that run's event log live, rendering the facts the worker measured. It resumes from its last cursor across a stream cut instead of replaying, and stops when the run reaches a state that appends nothing further. |
| `mr-data status RUN_ID --hosted` | Report the state Studio holds for one run, and with `--receipts` its execution summary and the sealed evidence for the Build it produced. |
| `mr-data deploy-dataset RUN_ID --hosted --output DIR` | Fetch that run's dataset through a short-lived signed URL, refuse it unless the bytes hash to the digest the release sealed, and write it once into `DIR`. |
| `mr-data acquire --hosted` | Fetch one public address in Studio's clean room and print the Receipt Studio bound, naming under `reader_binding` which party resolved the pinned Reader. A source that refused answers `status: "refused"` with its own sealed code and exits 1. |
| `mr-data acquire-cancel --workspace-id WORKSPACE_ID --crawler-session-id CRAWLER_SESSION_ID --hosted` | Cancel one still-active public acquisition using the coordinate `acquire` printed. By default it writes a private, per-coordinate receipt journal under the local mr-data state directory; a transport-loss retry reuses its idempotency key, and success prints Studio's exact cancellation receipt. Use `--journal FILE` only to override that location. |
| `mr-data approve --hosted` / `mr-data recipe-approve --hosted` | Record the decision you are asking for against one approval Studio holds, print the address where the person who must grant it does so, then wait and read the decision receipt back. The command never decides, and exits 2 unless a human recorded the decision that was asked for. |
| `mr-data note --run RUN_ID --hosted` | Append one cell of the operator-facing decision record to that run's own log, where the rendered notebook and the live view both read it: `--heading` is its one outcome-led line, the body arrives on standard input or from `--markdown-file`, and `--checkpoint` places it against the run-event it explains. Writing the same `--cell-id` again revises the cell in place. A cell is unsealed prose and is never attestation evidence. `--list` reads them all back. |
| `mr-data research-open --dataset DATASET_ID --question QUESTION_ID --source SOURCE_ID` | Open one research session and print each state as its worker warms up: `opening`, `warming`, `ready`. A workspace already holding its bound of live sessions is refused with the retry window Studio stated, never waited through. |
| `mr-data research-probe SESSION_ID --kind KIND --request JSON` | Ask the session's warm worker one question about a source, checked against the probe vocabulary before it is sent, follow the session run's stream until the probe settles, and print the answer read from the probe record. Exits 0 only for an answered probe; a probe answer is exploration and never evidence. |
| `mr-data research-status (SESSION_ID \| --run RUN_ID) [--probe PROBE_ID]` | Report Studio's own reading of one session, and of one probe with its answer. `--run` resolves the owning session directly in Studio; it never scans sessions client-side. |
| `mr-data research-close (SESSION_ID \| --run RUN_ID)` | Close the session, cancelling any probe still queued, so its worker exits. `--run` asks Studio to resolve and close the owning session directly. |

## Current local workflow

The supported local workflow is:

```text
question and output grain
  -> selected sources and rights evidence
  -> declarative dataset plan
  -> confined source snapshots
  -> deterministic transform and validation
  -> immutable Parquet Build
  -> two assigned reviews
  -> Checker-signed decision
  -> reviewed_candidate | fixes_required
  -> Studio source staging and immutable Recipe proposal
  -> authenticated Editor confirmation of the internal approval record
  -> Dataset activation Run and Cloud Current binding
```

For the controlled acceptance walkthrough that starts from a changing synthetic public CSV and,
after the hosted rollout is separately marked ready, ends at Cloud Current after a scheduled
Dataset Version, see [Local Build to Cloud refresh demo](docs/DEMO-REAL-REFRESH.md). Local authoring
does not require Docker; governed Linux execution remains in Studio.

The local engine supports string trimming, exact empty-string-to-null conversion, collision-safe
rename, closed-type casts, stable left equality joins, output selection, and deterministic quality
checks. `local-plan.v2` runs a wider closed set of ordered operations over the same shared
arithmetic: it adds exact decimal division, wide-to-long reshaping of columns the plan names, inner
and anti joins, and bounded text normalization for keys that will not otherwise match. Every one of
those is admitted by a written test that separates cleaning from modeling choices, and the same
test is what refuses a general window, a filled-in value, and the reverse of the reshape. See
[Transforms a plan may perform](docs/TRANSFORMS.md).

`local-plan.v2` also carries one retired operation, `prediction_label@1.0.0`. It named a value at a
future date, and the horizon it required is a modeling choice rather than a property of the
sources, so no newly authored plan may name it: authoring, plan checking, validation, export and
approval all refuse one that does. It remains readable so that a Build sealed with it still
verifies and a Recipe approved with it still runs. See
[Prediction labels](docs/PREDICTION-LABELS.md).

The installed stock source-to-Reader path supports certified Readers for local files,
credential-free public HTTPS sources, and bounded ordered public HTTPS collections. Each HTTPS
retrieval, Reader decode, and parse shares one attested clean-room operation; JSON and NDJSON
projection, ZIP, TAR, GZIP, XLSX, delimited text, and single-record GRIB2 are available under their
exact Reader settings and budgets. A collection can combine 1–16 explicit objects or a fixed
page-number range when every member uses the same Reader and normalized schema; see
[Deterministic source collections](docs/SOURCE-COLLECTIONS.md).

The review runtime handles initial Builds only. It records whether the Build passed review or needs
fixes, rejects nonzero failed-cycle counts, and does not verify repaired successors or derive a
human-direction status.

The package also contains a pure canonical empirical source-observation chain and deterministic
cadence inference with shared cross-system vectors. Both governed paths record that evidence
through one emitter: a hosted probe submits it to Studio, and `mr-data acquire --cadence-history`
records a local probe in an append-only folder. For one acquisition outcome the two produce the
same document byte for byte. It does not yet schedule probes or claim that a manually selected cron
was learned; Studio owns persistence and adaptive scheduling.

The weather Reader opens one extracted GRIB2 record. It refuses a full GFS or HRRR file containing
a collection of hundreds of records.

## Recipes

`frozen-recipe.v1` binds the question, requirements, selected source evidence, connector versions,
query templates, credential-reference names, transformation plan, validation policy, Bookmark
strategy, and engine versions. Its digest is lowercase SHA-256.

The checked-in example contains `recipe.json` and `recipe-execution.json`:

```bash
uv sync --locked --extra dev
mkdir -p .work/recipe
uv run --extra dev mr-data recipe-export \
  --recipe examples/city-temperature/recipe.json \
  --output .work/recipe/frozen-recipe.json
uv run --extra dev mr-data recipe-approve \
  --recipe .work/recipe/frozen-recipe.json \
  --approval-id approval-city-temperature-v1 \
  --approved-by local-operator \
  --approved-at 2026-07-19T12:00:00Z \
  --approval-policy-digest 0000000000000000000000000000000000000000000000000000000000000000 \
  --output .work/recipe/recipe-approval.json
uv run --extra dev mr-data recipe-run \
  --recipe .work/recipe/frozen-recipe.json \
  --approval .work/recipe/recipe-approval.json \
  --execution examples/city-temperature/recipe-execution.json \
  --input-root examples/city-temperature \
  --output .work/recipe/version-1 \
  --producer-principal local-recipe-producer \
  --producer-attempt initial-v1
uv run --extra dev mr-data recipe-verify .work/recipe/version-1
```

The output parent must exist. Export, approval, and run commands do not overwrite existing output.
Refresh and backfill executions require `--previous-run-dir` and the exact predecessor Build.

## Authority and security

- External agents may interpret questions, propose sources, author declarative inputs, and report
  findings. They do not calculate authoritative metrics or approve releases.
- The deterministic worker owns confinement, budgets, snapshots, transforms, checks, lineage,
  hashes, and artifact sealing.
- The trusted coordinator owns roots, assignments, attempt identities, enrolled signing keys, and
  immutable cycle lineage.
- The Harness owns deterministic public-catalog provider semantics, including Data.gov endpoint,
  pagination, catalog identity, revisions, normalization, and provenance. Catalog discovery never
  authorizes or acquires the underlying dataset resource.
- Studio owns hosted contracts, orchestration, persistence, scheduling, credentials, approval, and
  release.

Source bytes and dataset plans are untrusted. The local-file worker accepts normalized relative
paths to regular, single-link CSV files below an approved input root. It rejects traversal,
absolute paths, symlinks, hardlinks, special files, input/output overlap, source mutation, and
budget violations.

Builds are content-addressed and non-overwritable. Canonical payloads exclude ambient clocks and
absolute filesystem paths. Model output and source content cannot grant human approval.

The package does not provide local publication authority, local release authority, modeling,
general feature engineering, prediction serving, authenticated source access, scheduling, or
arbitrary code execution from an agent response. The one narrow exception to general feature
engineering is deterministic, terminal-only prediction-label construction described above.

## Hosted workers

`mr-data export-hosted-candidate` verifies a local Build and writes the canonical hosted
Builder-input envelope:

```bash
mr-data export-hosted-candidate /absolute/path/to/run \
  --output /absolute/path/to/new-candidate-input.json \
  --json
```

The hosted executables are:

- `mr-data-bootstrap`: exchanges workload identity for one Studio job and starts the fixed role.
- `mr-data-producer`: reconstructs and verifies a Build, then uploads reservation-bound artifacts.
- `mr-data-verifier`: independently verifies artifacts and submits a signed report.
- `mr-data-crawler-job`: performs allowlisted public acquisition and passes bounded bytes to a
  networkless Clean room.
- `mr-data-ingestion-job`: redeems one opaque workload-identity-bound ingestion dispatch and runs
  only a reviewed bounded delivery processor; see [Hosted ingestion delivery](docs/INGESTION-DELIVERY.md).

These commands cannot select or release a Build, create a dataset version, advance `latest`, or
access Studio databases, queues, secrets, or storage metadata directly.

## Verification

For ordinary edit-loop feedback, run `make test-fast`. This is a non-authoritative test tier that
omits semantically expensive scale, isolated-install, and live-repository-scan coverage;
`make verify` is required before review or merge.

```bash
uv sync --extra dev
make verify-offline-slice
make verify-hosted-workers
make verify-viewer
make verify-viewer-browser
make verify
```

`verify-hosted-workers` requires a sibling `mostlyright-studio` checkout. Set
`STUDIO_ROOT=/absolute/path` when it is elsewhere.

`make verify-thin-client` proves the hosted profile the way a stranger meets it: it builds the
base wheel, installs it into a fresh environment from an empty cache, refuses any compiled
extension it finds there, and runs the hosted command surface. The same body runs in CI as the
reusable `thin-client-smoke` workflow, fanned out over GitHub-hosted ubuntu, macOS and Windows:
one `py3-none-any` wheel is built once and installed by all three legs, on every pull request and
nightly.

There is no documented known-failing set; every failing node is a verification failure.

## Documentation

- [Architecture](docs/ARCHITECTURE.md)
- [Offline Vertical Slice](docs/OFFLINE-VERTICAL-SLICE.md)
- [Recipes](docs/FROZEN-RECIPES.md)
- [Sources and Acquisition](docs/SOURCES-AND-ACQUISITION.md)
- [Empirical source cadence](docs/SOURCE-CADENCE.md)
- [Readers](docs/READERS.md)
- [Deterministic source collections](docs/SOURCE-COLLECTIONS.md)
- [Notebook View](docs/NOTEBOOK-VIEW.md)
- [Deployment request](docs/DEPLOYMENT.md)
- [Local Build to Cloud refresh demo](docs/DEMO-REAL-REFRESH.md)
- [Hosted parity](docs/HOSTED-PARITY.md)
- [Catalogue release channel](docs/CATALOG-CHANNEL.md)
- [Hosted workers](docs/HOSTED-WORKERS.md)
- [Hosted worker publication](docs/HOSTED-WORKER-PUBLICATION.md)
- [Research-session worker](docs/SESSION-WORKER.md)
- [Transforms a plan may perform](docs/TRANSFORMS.md)
- [Review policy](docs/REVIEW-LOOP.md)
- [Contract ownership](docs/CONTRACTS.md)
- [Engine governance](docs/ENGINE-GOVERNANCE.md)

`.planning/` contains repository work records. Its contents are not product contracts, merge
authorization, approval, or release evidence.
