Metadata-Version: 2.4
Name: goesb-runner
Version: 0.8.8
Summary: Official Open Edge Speech Benchmark (GOESB) runner
Requires-Python: >=3.11
Requires-Dist: cryptography>=42.0
Requires-Dist: jsonschema>=4.21
Requires-Dist: packaging>=23.0
Requires-Dist: psutil>=5.9
Requires-Dist: pydantic>=2.6
Requires-Dist: pyyaml>=6.0
Requires-Dist: questionary>=2.0
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: mypy>=1.9; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff~=0.16.0; extra == 'dev'
Provides-Extra: faster-whisper
Requires-Dist: faster-whisper>=1.0.0; extra == 'faster-whisper'
Provides-Extra: mozilla-data-collective
Requires-Dist: datacollective>=0.5.0; extra == 'mozilla-data-collective'
Provides-Extra: vosk
Requires-Dist: soundfile>=0.12; extra == 'vosk'
Requires-Dist: vosk>=0.3.44; extra == 'vosk'
Provides-Extra: whisper-cpp
Requires-Dist: pywhispercpp>=1.5.0; extra == 'whisper-cpp'
Requires-Dist: soundfile>=0.12; extra == 'whisper-cpp'
Description-Content-Type: text/markdown

# GOESB Runner

The official, reproducible benchmark runner. It loads a signed **Benchmark
Profile** and **Benchmark Pack**, captures the full hardware/software/model
environment, runs the benchmark (batch / streaming / conversation) against a
pluggable **runtime adapter**, computes metrics, and emits a signed,
hash-verifiable result.

The runner **never executes arbitrary user code** (see the security model).

## Quick start
```bash
pip install -e ".[dev,faster-whisper]"
goesb --help
goesb env          # print captured environment fingerprint
goesb validate profiles/whisper-medium-en-batch/profile.yaml
goesb validate packs/librispeech-en/pack.yaml

# Fetch this pack's audio (not committed to git — FR-3.5) before running
# against it. Reruns are safe with --skip-download.
python ../scripts/fetch_librispeech_subset.py

# Run a batch benchmark end-to-end: environment capture, transcription,
# normalization + WER/CER/RTF/CPU/RAM, a signed+hashed result on disk.
goesb run whisper-medium-en-batch librispeech-en --repeats 2
```

Pass `--model-override tiny` (or `base`) to `goesb run` for a fast local smoke
test instead of downloading the full `whisper-medium` the profile specifies —
results from an override are for pipeline verification only, never comparable
to the official profile's numbers.

Schema validation currently requires running from within this monorepo
checkout (see `oesb_runner.schema_validation`); standalone packaging of the
schemas is tracked for M2.
