# hotato

> An open (MIT), offline, zero-install tool that scores voice-agent turn-taking
> from a call recording. Does your agent drop the turn, or hog it? It scores the
> timing from a recording and points each failure at a concrete fix. Runs entirely
> on your machine. Built to be called by an AI agent mid-task, as a CLI or a
> one-tool MCP server.

## Run it (zero-install)

- The 5-minute path: `uvx hotato doctor --stereo call.wav` scores the recording,
  writes the visual HTML report, opens it. `uvx hotato doctor` runs the bundled
  self-test instead.
- See a catch: `uvx hotato demo` scores the packaged intentionally failing
  battery and opens its report (exit 0 by design; `--fail` returns the real
  regression exit code).
- Score the bundled battery: `uvx hotato run --suite barge-in`
- Score one recording (two-channel WAV, caller and agent on separate channels):
  `uvx hotato run --stereo call.wav --stack livekit --format json`
- Visual report (self-contained HTML, per-event SVG timelines, analytics,
  per-frame inspector, print CSS): `uvx hotato report --stereo call.wav --out report.html`.
  `--format md` for Markdown; `--base base.json` for regression deltas.
- Team trends over many runs (pass rate, mean/median/p90 talk-over and
  time-to-yield, most common failure class): `uvx hotato team runs/ --html team.html`
- Research CSVs (events.csv, frames.csv, envelope.json, columns documented
  in-file): `uvx hotato export --stereo call.wav --out research/`
- Pytest plugin, auto-registered on install: a `hotato_score` fixture, plus
  `pytest --hotato-suite` to fail the session on a regression
  (`--hotato-suite-scenarios DIR --hotato-suite-audio DIR` for your own sets).
- As a one-tool MCP server (stdio): `uvx --from "hotato[mcp]" hotato-mcp` exposes
  exactly one tool, `voice_eval_run`, returning the identical JSON envelope.
  Optional `report_path` also writes the HTML report; the envelope carries the path.
- Tiered corpus suites (silver, silver-defects, gold, gold-defects; defect suites
  exit 1 by design): `hotato run --suite barge-in --scenarios corpus/suites/gold/scenarios --audio corpus/suites/gold/audio`
- PR check: copy `.github/workflows/hotato.yml` for a sticky results comment and
  a regression gate on every pull request.

## What it returns

A single JSON envelope (schema: `hotato/schema/envelope.v1.json`, `schema_version`
"1", additive-only). Per event, three objective timing signals: `did_yield`,
`seconds_to_yield`, `talk_over_sec`, plus a `signals` bus (`barge_in`, `latency`)
and a `verdict`.

Exit codes: `0` all pass, `1` a regression, `2` usage or IO error, or a single
recording that is not scorable. An event that cannot be judged (silent caller,
or agent silent at the caller onset) carries `scorable: false` and a plain
`not_scorable_reason`; it never surfaces as a normal pass or fail. Every failing
event carries a `fix`:

- `fix_class: "config"`: a concrete knob for your stack (livekit, pipecat, vapi,
  generic) with the direction and the honest trade-off.
- `fix_class: "engagement-control"`: a discrimination problem a single sensitivity
  dial cannot solve. Points high-level and vendor-neutral at the kind of fix it
  needs: a learned engagement-control / addressee-detection layer, an open
  research problem.

## Scope and method (each limit is a property of how it works)

- Reproducible timing against published thresholds. Every number is a measurement
  against a documented, overridable parameter, with an explicit ceiling; every
  frame is inspectable via `--dump-frames`.
- Energy over time: identity-blind and content-blind by construction. No speaker
  identification, diarization, transcription, or emotion detection.
- Two channels are ground truth for overlap. Mono runs with an onset label and
  marks its overlap as an estimate.
- The bundled fixtures are synthetic: a deterministic floor and a regression
  guard. Real validity comes from your own labelled calls.

- `hotato benchmark --stack S --recordings DIR` scores your captured battery; `benchmark compare a.json b.json` renders side-by-side deltas.

## Links

- README.md: quickstart and scope.
- METHODOLOGY.md: exactly how every number is computed, including the mean,
  median, and p90 definitions (p90 is linear interpolation; rates are fractions,
  never a percentage).
- docs/REPORTS.md: doctor, report, team, and export in depth.
- docs/PYTEST.md: the pytest fixture and the suite gate.
- docs/SUITES.md: the tiered corpus suites and the deterministic builder.
- docs/CI.md: the GitHub PR check and the sticky results comment.
- docs/SUBMITTING.md: contribute a consented, labelled real call.
- CONTRIBUTING.md, docs/CORPUS-GOVERNANCE.md: contribute real labelled fixtures.
- LICENSE: MIT. The open core is never relicensed.
