Metadata-Version: 2.4
Name: ask-monarch
Version: 0.1.31
Summary: Thin CLI for the hosted Ask Monarch source plane
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# Ask Monarch

## Teammate Setup

Install Ask Monarch for a teammate with one command. Get a current setup code from Josh or an Ask Monarch admin (don't reuse a code pasted into docs):

```bash
uvx ask-monarch setup --code <YOUR-SETUP-CODE>
```

The code is stable until Josh or an admin rotates it. This command installs or
refreshes the persistent CLI, exchanges the setup code, configures Codex and
Claude Code, runs doctor, runs the canonical verification query, and reports
the source id.

Ask Monarch is the conversational front door to its memory and capabilities.

A teammate should be able to ask Monarch what it knows, inspect the evidence, challenge the answer, and improve the memory as new artifacts appear.

Under the hood, Ask Monarch does not ask the live bucket or a loose pile of
files every time. It follows the source contract:

```text
map the source -> access it -> enumerate it -> parse it into atomic units -> verify receipts -> make it queryable
```

Once a source passes that gate, Ask Monarch answers from the parsed indexes with provenance. That is what makes answers fast, repeatable, and inspectable.

Read [ASKMONARCH-SPEC.md](ASKMONARCH-SPEC.md) for the deeper product contract and future direction.

## Architecture

Ask Monarch is artifact-first: raw company artifacts flow through the
`monarch-source` contract into a searchable store. Bucket refreshes also build a
canonical experiment register so questions about what ran do not have to infer
experiments from files. The `ask-monarch` CLI is the supported ask path.

![Ask Monarch architecture](docs/ask-monarch-architecture.png)

Runtime: the GitHub repo `manintheandes/ask-monarch` is deployed to the
`ask-monarch-server` VM; the CLI runs on the user or agent machine and calls the
hosted API on that VM.

## Visual Explainers

Ask Monarch's architecture and assay explainers are repo-owned visual assets.
Manim scenes live in [visuals/manim](visuals/manim). Remotion demo videos live
in [videos](videos). Both are scoped away from the hosted API and CLI runtime.

```bash
cd visuals/manim
uv run manim -pql scenes.py AskMonarchArchitecture
```

For assay visuals, start with Manim's plotting examples and map source-backed
rows onto axes, curves, markers, guide lines, and highlighted regions. See
[visuals/manim/README.md](visuals/manim/README.md).

For the current visual evidence loop, run the root verifier:

```bash
python3 scripts/verify_visual_evidence.py
```

That checks the source-action hook, overlay resolver, assay-overlay CLI dry-run,
Manim brand palette, reusable Manim assay templates, and a smoke still render.

For Remotion video work, use the repo-owned brand tokens in
[videos/src/brand.ts](videos/src/brand.ts) and validate with
`npm run validate:brand` from the `videos` folder.

## Current State

Ask Monarch currently has twelve verified source lanes. For the fuller
human-readable source list, see [docs/source-lanes.md](docs/source-lanes.md).

| Source lane | What it covers | Status |
| --- | --- | --- |
| `bucket` | `gs://monarch-videos-new`, including experiment outputs, CSV rows, spreadsheets, images, videos, PDFs, model metadata, structured sidecars, and a canonical experiment register spanning mapped DART, contact/no-contact, phytotoxicity, oviposition, post-exposure, spatial-affect, and arm-in-cage families | Parsed and queryable |
| `presentations` | 46 Monarch update decks from the `Storyboards_Presentations` Google Drive folder, scoped to January 1, 2026 onward | Parsed and queryable |
| `company_profile` | Monarch mission, problem, status quo, approach, headquarters, organisms, and crop-testing context | Parsed and queryable |
| `people_directory` | Monarch employees, advisors, and partnership facts | Parsed and queryable |
| `company_context` | Company context from `Monarch_for_askmonarch.pdf`, parsed into PDF pages, text blocks, and embedded-image references | Parsed and queryable |
| `investor_faq` | Investor-facing FAQ from `Monarch Investor FAQ 2.pdf`, parsed into PDF pages, text blocks, and embedded-image references | Parsed and queryable |
| `teams_recaps` | Recurring `Monarch R&D`, `Monarch Weekly`, and `Monarch Team Sync` Teams meeting transcripts, refreshed daily after midnight | Parsed and queryable |
| `scientific_refs` | Scoped scientific reference library, including chemical ecology, receptor/pathway, modeling, assay-prototype, scalability, EPA registration-support, and Teams-discussed journal-article material | Parsed and queryable |
| `milestones_doc` | `monarch_milestones` Google Doc, scoped to one fetched revision and parsed into paragraph units | Parsed and queryable |
| `compound_inventory` | Monarch compound / chemical inventory workbook from the user-provided `Updated Chemical Inventory.xlsx` attachment | Parsed and queryable |
| `chemistry_analysis` | `gs://monarch-chemistry-analysis`, including chemical activity registry rows, docking result rows, JSON score sidecars, protein-structure metadata, model metadata, images, scripts, and text sidecars | Parsed and queryable |
| `ask_insects_context` | Hash-pinned immutable public Ask Insects package with exact-species profiles, deep insect-knowledge domains, assay interpretation boundaries, public evidence, and explicit knowledge gaps; it is not digitally signed | Parsed and queryable |

### Experiment Register

The bucket index represents each real experiment or recorded trial once in
`experiments`. Tube, replicate, timepoint, treatment, measurement, video, and
processed-output evidence is attached beneath that row in:

- `experiment_observations`
- `experiment_treatments`
- `experiment_results`
- `experiment_artifacts`
- `experiment_register_coverage`

`run_date` follows strict source precedence: an explicit date in the family's
canonical ledger or metadata workbook, then embedded video acquisition time,
then an acquisition timestamp encoded in a raw filename or experiment folder.
If none is available, `run_date` remains null and the register records a source
gap. Bucket `time_created` and `updated` values never become run dates.

Upload and processing questions use the separate `file_activity` view. Run-date
inventory questions use `bucket_experiment_register_run_inventory_sql` with
answer shape `experiment_register_run_inventory`. That single query returns
one row per experiment with treatment or compound, dilution, solvent, control
status, every actual result and its status, media status, and exact experiment
and child provenance.
A validated response is answer-ready and must not trigger follow-up SQL. Agent
fast paths use `ask-monarch query 'QUESTION' --limit 100 --format markdown`,
which validates the same one hosted response and emits only the final compact
numbered table. Shared source text appears once while every row retains its
exact locator. The raw JSON route remains available for programs and
diagnostics but is too large for the user-facing Codex fast path.
Recent-compound questions
that also request the best result and its video use
`bucket_experiment_register_compounds_media_sql` with answer shape
`experiment_register_compounds_with_winner_media`.

That combined route lists every non-control compound in the requested run-date
window, ranks only comparable completed measurements, and links media through
the winning `experiment_id`. If no comparable result exists, Ask Monarch says
that no winner can be measured yet and returns no arbitrary video.

Two optional high-value release diagnostics are exact. A valid result for `what
experiments were run yesterday` returns complete deduplicated register rows for
the resolved absolute date from one hosted source call and renders visibly in
Codex in under 30 seconds. A valid result for `list the compounds that were
tested in the last 7 days and pull up the video with the most repellent compound
- all in-line` keeps the compound list, date window, ranking status, and media
obligation together and does not select a video when indexed results do not
establish a winner. Neither diagnostic is a completion gate.

Run the fresh-Codex diagnostic through the Codex desktop app's
bundled runtime. Low reasoning is deliberate here: the hosted response is
already validated and Codex only has to return it unchanged.

If you choose to run the first optional diagnostic, create a brand-new one-turn
Codex desktop task in this repository with the exact question `what experiments
were run yesterday`. After it finishes,
export that task with `codex_app` `read_thread`, setting
`includeOutputs=true`, `maxOutputCharsPerItem=20000`, and `turnLimit=1`. Save
the complete raw JSON result without editing it in a current-user-owned regular
file with mode `0600`. The visible verifier rejects a transcript copied from
the UI, missing command output, truncated output, older task history, and a
headless-only run.

```bash
git fetch origin
REVISION="$(git rev-parse HEAD)"
test "$REVISION" = "$(git rev-parse refs/remotes/origin/main)"
test -z "$(git status --porcelain=v1 --untracked-files=all)"
RELEASE_RECEIPT="$HOME/.codex/evals/ask-monarch/release-receipts/$REVISION.json"
DESKTOP_EVIDENCE="$HOME/.codex/evals/ask-monarch/desktop-task-$REVISION.json"
test -f "$DESKTOP_EVIDENCE"
chmod 0600 "$DESKTOP_EVIDENCE"
PINNED_PYTHON="$HOME/.local/share/uv/python/cpython-3.12.13-macos-aarch64-none/bin/python3.12"
test "$(/usr/bin/shasum -a 256 "$PINNED_PYTHON" | /usr/bin/awk '{print $1}')" = \
  "e2605291e058fdbe3102e8185d0ac5fe0e063398de617010a6af3a42a78f05e3"

"$PINNED_PYTHON" -I -S -B scripts/verify_visible_yesterday_experiments.py \
  --release-receipt "$RELEASE_RECEIPT" \
  --desktop-evidence "$DESKTOP_EVIDENCE" \
  --report "$HOME/.codex/evals/ask-monarch/visible-$REVISION.json"
```

Run that command from a completely clean checkout whose `HEAD` and fetched
`origin/main` both equal the revision named by the recent successful production
receipt. Copy the receipt without editing it and keep the local file non-writable
by group or other users. The verifier derives the expected runtime bundle and
installed CLI package from those two release artifacts. It does not accept
caller-provided executable hashes, package hashes, a different Codex binary, a
different Ask Monarch executable, a different latency limit, or a stale expected
row count.

The verifier always uses the Codex desktop app's bundled binary, the canonical
`~/.local/bin/ask-monarch` derived from the current account record, and the
system `/bin/zsh`. It times the whole Codex turn, audits actual hosted calls, and
parses the final visible table. It also binds its sealed headless route
diagnostic to the raw one-turn desktop export. CLI-only or headless-only latency
is not accepted as a proxy for what the user saw.

Current presentation receipt:

- 46 decks
- 622 slides
- 1,388 slide text or notes units
- 736 media/chart/link references
- 0 explicit gaps

The bucket receipt currently proves all live bucket objects are parsed and that
CSV rows are materialized atomically in SQLite. The large CSV-row count is not an
animal count; it is source-unit count from processed frame/result files.

Current Teams recap receipt:

- 24 meeting instances across R&D, Weekly, and Team Sync through May 12, 2026
- 24 matching transcript files
- 12,595 speaker/timestamp/utterance units
- 0 explicit gaps

Current company profile receipt:

- 7 company profile units
- 0 explicit gaps

Current people directory receipt:

- 15 people, advisor, and partnership units
- 0 explicit gaps

Current company context receipt:

- 22 PDF pages
- 89 text blocks
- 59 embedded-image references
- 171 total atomic units
- 0 explicit gaps

Current milestones doc receipt:

- 1 Google Doc revision
- 30 paragraph units
- 0 tables
- 0 comments
- 0 explicit gaps

Current compound / chemical inventory receipt:

- 8 workbook sheets
- 1,806 rows with values
- 9,280 non-empty cell units
- 0 explicit gaps
- Original SharePoint link access still failed for the current connector identity; the parsed source is the attached workbook copy.

Current chemistry analysis receipt:

- 11,302 bucket objects
- 550,576 CSV row units
- 30,412 JSON units
- 99,206 text line units
- 3,637 protein-structure metadata units
- 0 explicit gaps

Current Braintrust receipt:

- Project: `Andes / ask-monarch`
- Project id: `28216333-66fb-42a6-923b-622bda7a2fcc`
- Query responses include `trace_id`, `trace_span_id`, `trace_project`, `trace_project_id`, and `trace_url` when `BRAINTRUST_API_KEY` is configured.
- Query traces include spans for `trace.user_question`, `trace.tool_call.1.ask_monarch_source_query`, `trace.selected_evidence`, `trace.source_receipts`, and `trace.final_answer`.

## How To Ask

In any capable coding or research agent, use the repo-owned Ask Monarch skill:

```text
what is the most effective compound against mosquitoes [$askmonarch](/Users/josh/.codex/skills/askmonarch/SKILL.md)
```

The user should not need to know which source, database, table, bucket, or deck
contains the answer. `$askmonarch` means: use the skill's routing rules, call
the `ask-monarch` CLI for evidence, answer from the returned rows, and say
clearly when the memory does not yet contain enough evidence.

Personal mailbox access is a live overlay, not shared company memory. A mapped
Josh-only Ask Monarch token can access the shared Monarch source plane and the
mapped `josh@monarchcrops.com` mailbox through Microsoft Graph; the shared/team
token cannot access personal mail, and personal mail is not indexed into shared
SQLite.

For direct shell use, the remote CLI calls the hosted Ask Monarch service:

```bash
ask-monarch health
ask-monarch summary
ask-monarch query "what are the most effective repellents?"
ask-monarch query "list the experiments from last week" --limit 10
ask-monarch query "what did today's presentation cover?" --source presentations --limit 8
ask-monarch presentation --latest --open
ask-monarch presentation --date 2026-01-02 --open
ask-monarch query "what did the latest R&D meeting cover?" --source teams_recaps --limit 8
ask-monarch query "what does Monarch do?" --source company_profile --limit 8
ask-monarch query "who works at Monarch?" --limit 50
ask-monarch query "what does the company context say about residues?" --source company_context --limit 8
ask-monarch search "2pp mosquito" --limit 5
ask-monarch search damage --source presentations --limit 5
ask-monarch search "spatial repellents" --source teams_recaps --limit 5
ask-monarch search "acetone" --source compound_inventory --limit 5
ask-monarch company-profile-search mission --limit 5
ask-monarch people-search advisor --kind advisors --limit 10
ask-monarch company-context-search glossary --limit 5
ask-monarch query "62 chemicals" --source milestones_doc --limit 5
ask-monarch milestones-search "62 chemicals" --limit 5
ask-monarch compound-inventory-search acetone --limit 5
ask-monarch search DEET --source chemistry_analysis --limit 5
ask-monarch sql "SELECT COUNT(*) AS rows FROM csv_rows" --source chemistry_analysis
ask-monarch search "oviposition" --source ask_insects_context --limit 5
ask-monarch interpret-experiment EXPERIMENT_ID
ask-monarch interpret-experiment EXPERIMENT_ID --hide-outcomes
ask-monarch csv-search IR3535 --name-like combined_results.csv --limit 5
ask-monarch sql "SELECT COUNT(*) AS slides FROM slide_units" --source presentations
ask-monarch --open-trace query "what does Monarch do?" --source company_profile --limit 5
ask-monarch trace-url TRACE_ID
ask-monarch personal-mail-status
ask-monarch personal-mail-search "mosquito field trial" --days 30 --limit 10
ask-monarch personal-mail-read GRAPH_MESSAGE_ID
ask-monarch personal-mail-draft --to person@example.com --subject "Subject" --body "Body"
ask-monarch personal-mail-reply-draft GRAPH_MESSAGE_ID --body "Reply body"
ask-monarch personal-mail-send-draft GRAPH_DRAFT_ID
```

The CLI returns JSON so Codex, Claude Code, shell scripts, and dashboards can
consume the same source evidence reliably.

## How Corrections Improve Ask Monarch

When Ask Monarch gives a weak answer, the fix should become part of the system,
not just part of one chat. There are three places to make that correction.

For source-question regressions, run the live drill:

```bash
python3 scripts/run_askmonarch_regression.py
```

The drill replays known tricky questions against the hosted source service and
checks both the returned source values and the `$askmonarch` skill rules that
should route future agents correctly. When a case fails, reproduce it manually,
patch the skill/source/parser/CLI layer that caused the miss, then add or
tighten the case before rerunning the drill.

Latency counts too. The drill prints wall-clock time per case, summarizes the
slowest case, and fails any case over the default per-case budget of 10 seconds.
Use `--default-max-seconds` to tune the budget for a run, or set `max_seconds`
on a case that deserves a tighter or looser threshold.

When intentionally failure-mining, use plausible teammate questions, not
impossible trivia. Count a failure when the hosted source path returns no
usable evidence, the route is ambiguous, units cannot be verified, or the answer
would require inventing a linkage. Each useful failure should leave behind a
durable rule, a regression case, or a named source/tooling gap.

For answer-quality regressions, run the live answer contract gate:

```bash
python3 scripts/verify_askmonarch_answer_quality.py
```

For a clean pass/fail check that does not rewrite the committed ledger outputs:

```bash
python3 scripts/verify_askmonarch_answer_quality.py --check-only
```

That gate checks that composed answers include the requested factual claims,
that evidence rows have citations, that broad questions cite multiple source
lanes when needed, and that negative/source-gap answers name the checked
sources instead of making absolute claims over unqueryable artifacts. It writes:

```text
evals/askmonarch_answer_quality_results.json
evals/askmonarch_answer_quality_findings.md
evals/askmonarch_answer_quality_ledger.html
```

Each ledger case must explain what failed, why it failed, how the correction was
made durable, and the exact verification command that proves the correction.
See `evals/askmonarch_quality_probe_summary_2026-05-03.md` and
`evals/askmonarch_quality_probe_summary_2026-05-03.html` for the May 3 hard
probe refresh summary.

### 1. Fix the agent instructions

Update the `$askmonarch` skill when Codex or Claude misunderstood the question
or chose the wrong source.

Example: if someone asks "how many mosquitoes have we used?" the skill should
say to use the assay ledger's `n_mosquitoes` field, not CSV frame rows.

Example: if someone asks "what compounds does Monarch have in inventory?", the
skill should route to `compound_inventory` and count or list `Master!D`
compound names, not bucket search snippets. If they ask "where is X?", return
the matched `Master` row fields such as Location/Bin, Vendor, CAS, Quantity,
SDS, and cell provenance.

Example: if someone asks "what experiments were run today?", the skill should
resolve today to an absolute date and query `experiments.run_date` across every
registered assay family. It should not include bucket uploads or processed files
unless the user asks about file activity.

### 2. Fix the shared rulebook

Update the README or spec when the rule should be visible to everyone working
on Ask Monarch.

Example: if presentations are queryable only from January 1, 2026 onward, the
spec should say that clearly so future agents do not treat every old deck as
source-grade.

### 3. Fix the tool

Add or improve an API/CLI command when the same question will come up again and
should not depend on a fresh ad hoc SQL query each time.

Example: if animal-use questions become common, add a command like
`ask-monarch animal-use mosquitoes` that always uses the right ledger field and
returns the right evidence.

For most Monarch teammates, the workflow stays simple: ask through Codex,
Claude, or another agent and inspect the evidence. Behind the scenes, the agent
uses the skill and `ask-monarch` CLI. Power users can also use the remote CLI
directly. Everyone benefits when recurring corrections are captured in these
shared layers.

## Hosted Service

Ask Monarch is deployed on a Google Compute Engine VM.

| Piece | Value |
| --- | --- |
| VM | `ask-monarch-server` |
| Google Cloud project | `gen-lang-client-0407939408` |
| Zone | `us-central1-a` |
| Main release | `/opt/ask-monarch/main/releases/$REVISION` |
| Main Python environment | `/opt/ask-monarch/main/venvs/$REVISION` |
| Experiment-review release | `/opt/ask-monarch/experiment-review/releases/$REVISION` |
| Legacy aggregate data root | `/home/josh/ask-monarch/artifacts` (data only, never served as code) |
| Public HTTPS endpoint | `https://ask-monarch.34.121.138.236.sslip.io` |
| Main local listener | `http://127.0.0.1:8787` |
| Review local listener | `/run/ask-monarch-experiment-review/review.sock` |
| Systemd services | `ask-monarch-http.service`, `ask-monarch-experiment-review.service` |

The services are read-only. Public traffic enters through HTTPS and
bearer-token auth. Nginx proxies normal routes to the private main listener and
only exact `POST /experiment-interpretation` to the isolated Unix socket.
Production code is installed as a root-owned immutable release. It is never
served from a mutable checkout.

The canonical deployment and proof sequence is
[docs/production-release.md](docs/production-release.md).

Health check:

```bash
curl -H "Authorization: Bearer $ASK_MONARCH_TOKEN" \
  https://ask-monarch.34.121.138.236.sslip.io/health
```

Unauthenticated requests should return `401`.

## Team Onboarding

Do not send broad team invites until the clean-teammate simulation passes and
the Codex app, Codex terminal, Claude app, and Claude Code terminal surfaces
are verified or explicitly recorded as manually confirmed.

Admin preflight:

```bash
python3 scripts/simulate_teammate_onboarding.py --mode temp-home
```

The simulator creates a fresh one-time setup code, installs into a fake clean
home, verifies Codex, shared `.agents`, and Claude setup, runs
`ask-monarch doctor`, runs source-plane parity checks, and asks canonical
Monarch questions for answer parity. It writes reports under
`output/onboarding-simulations/`.

For each teammate, create one personal setup code:

```bash
ask-monarch onboarding-code create --email person@monarchcrops.com --label "Person setup" --ttl-hours 168
```

Send that teammate only their own install command:

```bash
ASK_MONARCH_SETUP_CODE='MNR-XXXX-YYYY' bash -c 'curl -fsSL https://ask-monarch.34.121.138.236.sslip.io/install.sh | bash'
ask-monarch doctor
```

After installation, they should open a fresh Codex or Claude thread before
asking Monarch questions.

## Source Inspection

You can also pull up assay videos during Q&A. A useful Ask Monarch answer should
not stop at a numeric result when the underlying assay is inspectable. When an
answer identifies an exact assay row, source video path, or filename, the CLI can
resolve the linked raw video from the verified bucket index, download it into the
local media cache, and open it:

```bash
ask-monarch assay-media --filename IMG_0049_Contact_3PP.mov --open
ask-monarch assay-media --row 1255 --source-name mosquitoes/Mosquito_DART_assay_Results_GCS.xlsx --sheet Assays --open
```

This is the explicit raw-media exception path. Ask Monarch still answers from
verified source indexes first; raw videos are opened only after the indexed
source record has been resolved. If a raw video is not linked yet, the command
returns diagnostics instead of pretending the video is available.

Assay metric overlays are the next inspectable layer. When an assay has processed
metrics, Ask Monarch should be able to move from:

```text
Peak mean avoidance was 0.683 at 05:30.
```

to:

```text
Peak mean avoidance was 0.683 at 05:30.
Source: advanced_metrics.csv#row=6.
Artifact: raw assay video with the avoidance curve overlaid.
```

The `assay-overlay` command does that for supported processed metric CSVs. DART
and related tube assays use `advanced_metrics.csv` with `T*_avoidance` columns.
Contact/non-contact repellency videos use `frame_metrics.csv` with `contact_pi`
and inside/on-band/outside counts. The command resolves the matching raw video
from the verified bucket index and renders a synced time-lapse overlay. The
answer source is still the verified SQLite row; the overlay is an inspection
artifact. Today this runs from the repo/VM environment where the verified SQLite
index is present. With `--open`, local assay videos and rendered overlays open
in a browser player.

```bash
ask-monarch assay-overlay \
  --metric-csv 'mosquitoes/processed/_2026_04_28_14_44_46_BHomocyclo3_mov_results/_2026_04_28_14_44_46_BHomocyclo3_mov_results/advanced_metrics.csv' \
  --output videos/out/mosquito-bhomocyclo-overlay.mp4 \
  --speed 15 \
  --open
```

The renderer automatically discovers `T*_avoidance` columns, narrows to the
assay tubes when source metadata declares them, resolves the raw video object,
downloads it into `videos/source/`, and prints a proof payload with the peak row,
timestamp, source CSV provenance, raw media URI, and output video probe.

Fast verification:

```bash
python3 scripts/verify_assay_metric_overlays.py
```

Two verified examples:

```bash
# Fly repellency: 2PP, treatment tubes T5-T8, peak 50:30 / 0.654
ask-monarch assay-overlay \
  --metric-csv 'flies/processed/2026-04-22 12-12-14_results/2026-04-22 12-12-14_results/advanced_metrics.csv' \
  --output videos/out/fly-2pp-overlay.mp4 \
  --speed 30 \
  --open

# Mosquito DART: BHomocyclo, tubes T1-T3, peak 05:30 / 0.683
ask-monarch assay-overlay \
  --metric-csv 'mosquitoes/processed/_2026_04_28_14_44_46_BHomocyclo3_mov_results/_2026_04_28_14_44_46_BHomocyclo3_mov_results/advanced_metrics.csv' \
  --output videos/out/mosquito-bhomocyclo-overlay.mp4 \
  --speed 15 \
  --open

# Fly non-contact repellency: 2PP, frame-level contact_pi
ask-monarch assay-overlay \
  --metric-csv 'contact_repel_videos/flies/processed/2PP_non_contact/frame_metrics.csv' \
  --output videos/out/fly-2pp-non-contact-overlay.mp4 \
  --speed 15 \
  --open
```

The R&D presentation lane is openable too. Indexed Monarch update decks start
with the January 2, 2026 presentation, and `ask-monarch presentation --open`
opens the matching Google Slides locator returned by the verified presentations
index:

```bash
ask-monarch presentation --latest --open
ask-monarch presentation --date 2026-01-02 --open
ask-monarch presentation --query "3-Step Process" --open
```

Every query row that has source provenance can also return `source_actions`.
Those actions are the provenance resolver layer: they do not change the answer
source, which remains verified SQLite3, but they tell a client how to inspect the
underlying artifact when one is available. A returned action can point to a raw
video, Google Slides deck or slide, transcript annotation, CSV row, workbook
cell, PDF page, image, assay metric overlay, or other artifact. Power users can
open any returned locator directly:

```bash
ask-monarch resolve-source --locator 'https://docs.google.com/presentation/d/.../edit#slide=2' --open
ask-monarch resolve-source --locator 'gs://monarch-videos-new/path/to/video.mov' --open
ask-monarch assay-overlay --locator 'gs://monarch-videos-new/path/to/advanced_metrics.csv#row=6' --open
ask-monarch assay-overlay --locator 'gs://monarch-videos-new/path/to/frame_metrics.csv#row=10' --open
```

The installer saves the token locally at `~/.config/ask-monarch/config.json`.
To reconfigure it later:

```bash
ask-monarch configure
```

See [docs/remote-cli.md](docs/remote-cli.md) for the full CLI reference.

## Agent Skills

This repo owns the agent skills/instructions that make the experience feel like
asking the company rather than running a database query:

```text
skills/askmonarch/SKILL.md
skills/braintrust/SKILL.md
skills/correction/SKILL.md
skills/monarch-source/SKILL.md
```

The hosted installer writes:

- Codex global instructions: `~/.codex/AGENTS.md`
- Codex skills: `~/.codex/skills/`
- Claude global instructions: `~/.claude/CLAUDE.md`
- Claude skill: `~/.claude/skills/ask-monarch/`

Install or refresh repo-owned Codex skills on the VM with:

```bash
python3 scripts/install_codex_skills.py
```

`monarch-source` defines when a Monarch artifact becomes source-grade.
`$askmonarch` is the conversational front door over source-grade Monarch memory.
`correction` defines what to do when an Ask Monarch answer needs a durable
routing or answer fix. `braintrust` defines the Ask Monarch trace inspection and
verification workflow. The canonical text is host-agnostic: Codex, Claude,
Gemini, Pi, OpenCode, and other agents should follow the same source-routing
contract when they can run the CLI or call the hosted service.

## Source Lanes

### Bucket

- Source id: `monarch_videos_new`
- Bucket: `gs://monarch-videos-new`
- Project: `gen-lang-client-0407939408`
- Parser: `scripts/source_pipeline.py`
- Verification: `scripts/verify_source_complete.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-videos-new/source_index.sqlite` (the lane directory resolves to the external large-artifact root)

Useful commands:

```bash
python3 scripts/parse_monarch_bucket.py --manifest-only
python3 scripts/source_pipeline.py init
python3 scripts/source_pipeline.py parse --lane structured
python3 scripts/source_pipeline.py parse --lane video
python3 scripts/source_pipeline.py parse --lane model
python3 scripts/source_pipeline.py parse --lane pdf
python3 scripts/source_pipeline.py parse --lane image
python3 scripts/source_pipeline.py build-derived
python3 scripts/source_pipeline.py status
python3 scripts/verify_source_complete.py
```

`build-derived` creates the canonical experiment register and its child
observation, treatment, result, artifact, and coverage tables across mapped
assay families. It also creates the separate `file_activity` view. A run date
comes only from the canonical ledger/metadata date, embedded source-video
acquisition time, or a timestamp encoded in the raw filename/folder. Missing
dates remain explicit source gaps.

`experiment_artifacts` is the canonical source-grade bridge from an exact
`experiment_id` to raw media and processed outputs. Legacy `assay_media_links`
remains available for row-based callers. The remote CLI exposes that older path as
`ask-monarch assay-media ... --open`, which can cache and open the video for
inspection during an Ask Monarch session.

### Presentations

- Source id: `monarch_storyboards_presentations_2026`
- Folder: `Storyboards_Presentations`
- Folder id: `1-dL59DdeoPySGbqE5cjGqednyp4CIzch`
- Boundary: Monarch update decks from January 1, 2026 onward
- Freshness: daily Drive enumeration and verified hosted promotion after the VM refresh is inactive
- Refresher: `scripts/refresh_monarch_drive_sources.py`
- Parser: `scripts/parse_storyboards_presentations.py`
- Query helper: `scripts/query_storyboards_presentations.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/storyboards-presentations-2026/source_index.sqlite`
- Receipt: [docs/storyboards-presentations-2026-source.md](docs/storyboards-presentations-2026-source.md)

Useful commands:

```bash
ask-monarch presentation --latest --open
ask-monarch presentation --date 2026-01-02 --open
ask-monarch presentation --query "3-Step Process" --open
python3 scripts/parse_storyboards_presentations.py
python3 scripts/query_storyboards_presentations.py 2PP --limit 5
python3 scripts/query_storyboards_presentations.py damage --limit 5
```

`ask-monarch presentation` resolves decks through the hosted presentations
source index and opens the Google Slides locator stored in deck or slide
provenance. This source does not claim the entire historical presentation folder. Older
decks, PDFs, notebooks, interviews, Teams messages, emails, academic papers, and
dashboards are future source candidates until they pass the same source gate.

### Monarch Teams Recaps

- Source id: `monarch_recurring_teams_recaps`
- Source lane: `teams_recaps`
- Source system: Microsoft Teams / Microsoft Graph
- Boundary: recurring `Monarch R&D` meetings from January 2, 2026 onward, `Monarch Weekly` meetings from May 5, 2026 onward, and `Monarch Team Sync` meetings from May 6, 2026 onward
- Freshness: VM refresh runs just after midnight America/Los_Angeles and includes meetings through the previous Pacific calendar day
- Refresher: `scripts/refresh_teams_recaps_source.py`
- Fetcher: `scripts/fetch_monarch_r_and_d_teams_recaps.py`
- Parser: `scripts/parse_monarch_r_and_d_teams_recaps.py`
- Query helper: `scripts/query_monarch_r_and_d_teams_recaps.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-r-and-d-teams-recaps/source_index.sqlite`
- Receipt: [docs/monarch-r-and-d-teams-recaps-source.md](docs/monarch-r-and-d-teams-recaps-source.md)

Useful production-safe commands:

```bash
sudo systemctl start ask-monarch-refresh.service
python3 scripts/query_monarch_r_and_d_teams_recaps.py "spatial repellents" --limit 5
```

The manual service start refreshes every scheduled VM lane through the shared
production lock. The Teams child refresher is an internal implementation detail
and must not be invoked directly against production.

This lane is meeting-transcript memory, not the whole Teams universe. Teams
messages, channels, company-wide email sources, papers, dashboards, and older
meeting transcripts remain future source candidates until they pass the same
source gate. Personal mailbox access, when mapped for a personal token, is a
private live overlay and does not promote those emails into shared company
memory.

### Company Profile

- Source id: `monarch_company_profile`
- Source lane: `company_profile`
- Raw source: `sources/monarch-company-profile/company_profile.json`
- Parser: `scripts/parse_company_lanes.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-company-profile/source_index.sqlite`
- Receipt: [docs/monarch-company-profile-source.md](docs/monarch-company-profile-source.md)

Useful commands:

```bash
python3 scripts/parse_company_lanes.py
ask-monarch company-profile-search mission --limit 5
ask-monarch search "where is Monarch based" --source company_profile --limit 5
```

This lane is durable company framing, not an automatically discovered corporate
knowledge base. Update the JSON source when the company framing changes, then
rerun the parser.

### People Directory

- Source id: `monarch_people_directory`
- Source lane: `people_directory`
- Raw source: `sources/monarch-people-directory/people_directory.json`
- Parser: `scripts/parse_company_lanes.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-people-directory/source_index.sqlite`
- Receipt: [docs/monarch-people-directory-source.md](docs/monarch-people-directory-source.md)

Useful commands:

```bash
python3 scripts/parse_company_lanes.py
ask-monarch people-search Avinash --limit 5
ask-monarch people-search advisor --kind advisors --limit 20
ask-monarch sql "select name, role, lane, provenance_grain from units where lane = 'employees' order by name" --source people_directory --limit 50
ask-monarch sql "select name, role, lane, provenance_grain from units order by lane, name" --source people_directory --limit 50
```

This lane covers people facts explicitly entered into the source JSON. It should
not be treated as a full HRIS, inbox, or LinkedIn scrape.

### Company Context

- Source id: `monarch_company_context`
- Source lane: `company_context`
- Raw source: `sources/monarch-company-context/Monarch_for_askmonarch.pdf`
- Boundary: exactly this one 22-page PDF file
- Parser: `scripts/parse_monarch_company_context.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-company-context/source_index.sqlite`
- Receipt: [docs/monarch-company-context-source.md](docs/monarch-company-context-source.md)

Useful commands:

```bash
python3 scripts/parse_monarch_company_context.py
ask-monarch company-context-search glossary --limit 5
ask-monarch search "USDA Insecticide Residue Report" --source company_context --limit 5
ask-monarch sql "select page_number, unit_index, text, provenance_grain from text_units where text like '%USDA%'" --source company_context --limit 20
```

This is company context sourced from PDF intake evidence. Current
employee/advisor questions should use `people_directory` first; older names in
the PDF do not override current people-directory truth.

### Investor FAQ

- Source id: `monarch_investor_faq`
- Source lane: `investor_faq`
- Raw source: `sources/monarch-investor-faq/monarch-investor-faq-2026-06-16.pdf`
- Boundary: exactly this one 5-page PDF file
- Parser: `scripts/parse_monarch_investor_faq.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-investor-faq/source_index.sqlite`
- Receipt: [docs/monarch-investor-faq-source.md](docs/monarch-investor-faq-source.md)

Useful commands:

```bash
python3 scripts/parse_monarch_investor_faq.py
ask-monarch investor-faq-search burn --limit 5
ask-monarch search "commercialization partner" --source investor_faq --limit 5
ask-monarch sql "select page_number, unit_index, text, provenance_grain from text_units where text like '%break-even%'" --source investor_faq --limit 20
```

This lane is investor-facing FAQ evidence. It is useful for FAQ claims on raise,
burn, break-even target, commercialization partner, distribution, product
results, market framing, risks, and partnership discussions. Current
employee/advisor questions should still use `people_directory` first unless the
user explicitly asks what the investor FAQ says.

### Milestones Doc

- Source id: `monarch_milestones_doc`
- Source lane: `milestones_doc`
- Source system: Google Docs / Google Drive connector
- Boundary: exactly the `monarch_milestones` Google Doc revision fetched on May 1, 2026
- Parser: `scripts/parse_monarch_milestones_doc.py`
- Query helper: `scripts/query_monarch_milestones_doc.py`
- Index on VM: `/home/josh/ask-monarch/artifacts/monarch-milestones-doc/source_index.sqlite`
- Receipt: [docs/monarch-milestones-doc-source.md](docs/monarch-milestones-doc-source.md)

Useful commands:

```bash
python3 scripts/parse_monarch_milestones_doc.py
ask-monarch query "phytotoxicity" --source milestones_doc --limit 5
ask-monarch milestones-search "62 chemicals" --limit 5
ask-monarch sql "select paragraph_index, text, provenance_grain from paragraph_units" --source milestones_doc --limit 5
```

This lane is a document-revision snapshot. Refetch the Google Doc and rerun the
parser before using it for claims about later milestone edits.

## Querying Locally

The hosted path is preferred for shared use. Local scripts are still useful for
debugging and parser development:

## Braintrust Tracing

Ask Monarch query requests emit Braintrust traces when the runtime has the
Python dependency and an API key:

```bash
python3 -m pip install -r requirements.txt
export BRAINTRUST_API_KEY=...
export ASK_MONARCH_BRAINTRUST_PROJECT=ask-monarch
export ASK_MONARCH_BRAINTRUST_PROJECT_ID=28216333-66fb-42a6-923b-622bda7a2fcc
```

Tracing is on by default for `/query` when the SDK and API key are present, and
fails open: if either is missing, source queries still run and simply omit trace
fields. Set `ASK_MONARCH_BRAINTRUST_TRACE_ALL=0` only for local debugging that
must suppress default traces. When tracing is active, every `/query` response
includes a Braintrust full trace URL. The local Braintrust context is stored in
`.bt/config.json`.

```bash
python3 scripts/query_source.py --summary
python3 scripts/query_source.py --q "Preference Index" --kind pdf_page --limit 5
python3 scripts/query_source.py --csv-q IR3535 --name-like combined_results.csv --limit 5
python3 scripts/query_source.py --sql "select name, row_number, row_json, provenance_grain from csv_rows_with_provenance where row_json like '%IR3535%' limit 5"
```

See [docs/querying-monarch-source.md](docs/querying-monarch-source.md).

## Deep Research Max

Normal Ask Monarch questions should query verified source indexes directly.
Deep Research Max is only for long-form external research, literature
comparison, scientific synthesis, market/context scans, or visual analysis
grounded in Monarch evidence.

Deep Research does this:

```text
research question -> verified multi-source evidence bundle -> Gemini Deep Research Max -> cited report
```

The bundle currently includes verified lanes such as `bucket`, `presentations`,
`teams_recaps`, `company_profile`, `people_directory`, and `company_context`
when relevant. The `milestones_doc` lane is queryable through the hosted HTTP
API and remote CLI, but should be added to Deep Research bundles before relying
on it in that mode.
If a required source lane is missing or has unverified receipts, Deep Research
stops before launch instead of producing a partial internal-evidence report.

Bundle-only:

```bash
python3 scripts/deep_research_monarch.py "Compare our 2PP fly and mosquito assay signal against published 2-propylphenol repellency literature."
```

Launch Gemini Deep Research Max:

```bash
GEMINI_API_KEY=... python3 scripts/deep_research_monarch.py "Compare our 2PP fly and mosquito assay signal against published 2-propylphenol repellency literature." --launch --wait
```

Outputs live under:

```text
reports/deep-research/
```

See [docs/deep-research-mode.md](docs/deep-research-mode.md).

## Daily Refresh

The VM should refresh parsed scheduled source indexes daily just after midnight
America/Los_Angeles. The VM-scheduled lanes are `bucket`
(`gs://monarch-videos-new`), `chemistry_analysis`
(`gs://monarch-chemistry-analysis`), and `teams_recaps` recurring Teams meeting
transcripts. A separate launchd job refreshes `presentations` and
`scientific_refs`, then waits for the VM refresh to become inactive before
promoting those Drive-backed indexes.

The main bucket path is incremental: reuse the current verified SQLite index,
enumerate the live bucket, invalidate only added/changed/deleted objects, parse
those objects, then verify before replacing the live SQLite files:

```text
live source -> staged incremental parse -> receipts/gaps -> verification -> atomic swap
```

Repo-owned pieces:

- `scripts/refresh_source_index.py`
- `scripts/refresh_chemistry_analysis_source.py`
- `scripts/refresh_teams_recaps_source.py`
- `scripts/refresh_bucket_sources.py`
- `scripts/refresh_monarch_drive_sources.py`
- `scripts/parse_scientific_reference_library.py`
- `deploy/systemd/ask-monarch-refresh.service`
- `deploy/systemd/ask-monarch-refresh.timer`
- `deploy/launchd/com.ask-monarch.drive-refresh.plist`
- [docs/daily-source-refresh.md](docs/daily-source-refresh.md)

## Operations

VM service commands:

```bash
sudo systemctl status ask-monarch-http.service
sudo systemctl restart ask-monarch-http.service
sudo journalctl -u ask-monarch-http.service -n 100 --no-pager
sudo systemctl status nginx
sudo nginx -t
```

Do not update production with a pull inside a long-lived VM checkout. Install
the exact remote `main` merge into revision-specific code and virtual
environment directories, move the active links atomically, and run the release
verifier. A successful deploy writes
`/var/lib/ask-monarch/release-receipts/$REVISION.json`. After that, publish and
refresh the installed CLI and skill copies, verify Codex, shared `.agents`, and
Claude parity, then run Reality Eval #4, the 50-question sole completion gate.
One unchanged 50/50 run passes it. Merge, deployment, required source refresh,
and installed-agent parity are prerequisites for that counted run. Recording,
privacy review, and sharing are its proof. The exact visible latency question
and fresh 200-case suite are optional supporting diagnostics. They can localize
defects, but they are not separate gates and do not block or complete the goal.
Follow [docs/production-release.md](docs/production-release.md) for the release
path and diagnostic commands.

## Repo Layout

```text
ASKMONARCH-SPEC.md              product/source contract
config/source-map.yaml          source registry and query-plane map
docs/                           operating docs and source receipts
scripts/                        parsers, query tools, HTTP service, CLI, refresh
skills/                         repo-owned Codex skills
deploy/systemd/                 VM service and refresh units
reports/deep-research/          generated Deep Research bundles and reports
```

Generated source-plane outputs are intentionally not committed to GitHub. Keep
the repo as code, specs, scripts, skills, and docs. Keep large parsed artifacts
on the VM or in artifact storage.

## Source Contract

A source is not queryable just because it exists. Ask Monarch treats a source as
queryable only when:

- it is listed in `config/source-map.yaml`
- it can be accessed through an authenticated route
- its live contents are enumerated
- its artifacts are parsed into atomic units
- each unit has source-grade provenance
- receipt and gap files are written
- verification passes
- query tools can return evidence with exact locators

That contract is the core of the system: do the hard parsing once, verify it,
store it in a parsed index, and query the index fast.
