# `orchestrator.pkg.accuracy`
<!-- generated by `orchestrator understand`; the PKG is source of truth, edits here are advisory -->

[← Episteme](../README.md) · [Architecture](../architecture.md)

**Source:** [`src/orchestrator/pkg/accuracy.py`](../../src/orchestrator/pkg/accuracy.py)

6 types · 17 functions · python

## Changing this safely

**Tested by** (2): `tests.pkg.test_accuracy`, `tests.pkg.test_scoreboard`

**Most depended-upon here** — a change to these reaches the most code (call graph, ≤4 hops):

- [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53) — reaches **11** symbols
- [`KindScore`](../../src/orchestrator/pkg/accuracy.py#L58) — reaches **7** symbols
- [`_edge_key`](../../src/orchestrator/pkg/accuracy.py#L151) — reaches **6** symbols
- [`_require`](../../src/orchestrator/pkg/accuracy.py#L136) — reaches **6** symbols
- [`measured_recall`](../../src/orchestrator/pkg/accuracy.py#L542) — reaches **6** symbols
- [`CaseReport`](../../src/orchestrator/pkg/accuracy.py#L80) — reaches **5** symbols
- [`ParityReport`](../../src/orchestrator/pkg/accuracy.py#L348) — reaches **5** symbols
- [`_case_batch`](../../src/orchestrator/pkg/accuracy.py#L232) — reaches **5** symbols

_1 of the symbols other code depends on here have no test path the graph can see — worth a second look before changing them. Absence of a path is not proof of absence of a test: calls through an attribute chain (`obj.method()`) are skipped rather than guessed at, so indirect coverage is invisible._

## Documented in

`docs/specs/build-documents/PKG-ACC-2-build.md#4-pkg-what-the-graph-knows`, `docs/specs/build-documents/PKG-ACC-3-build.md#4-pkg-what-the-graph-knows`

## Types

### `AccuracyReport`

[`src/orchestrator/pkg/accuracy.py:96`](../../src/orchestrator/pkg/accuracy.py#L96)

- **Called by** (1): [`score_corpus`](../../src/orchestrator/pkg/accuracy.py#L307)
- **Fields**: `cases`, `skipped`
- **Documented in**: `docs/specs/build-documents/PKG-ACC-1-build.md#7-files`

### `CaseReport`

[`src/orchestrator/pkg/accuracy.py:80`](../../src/orchestrator/pkg/accuracy.py#L80)

- **Called by** (1): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Fields**: `case`, `declared_false_positives`, `edges`, `known_gaps`, `language`, `missing`, `nodes`, `provenance_checked`, `provenance_drift`, `unlabelled`

### `CorpusError`

[`src/orchestrator/pkg/accuracy.py:53`](../../src/orchestrator/pkg/accuracy.py#L53)

- **Extends** (1): `Exception`
- **Called by** (6): [`_edge_key`](../../src/orchestrator/pkg/accuracy.py#L151), [`_load_case`](../../src/orchestrator/pkg/accuracy.py#L160), [`_node_key`](../../src/orchestrator/pkg/accuracy.py#L142), [`_require`](../../src/orchestrator/pkg/accuracy.py#L136), [`score_corpus`](../../src/orchestrator/pkg/accuracy.py#L307), [`score_parity`](../../src/orchestrator/pkg/accuracy.py#L382)

### `KindScore`

[`src/orchestrator/pkg/accuracy.py:58`](../../src/orchestrator/pkg/accuracy.py#L58)

- **Called by** (2): [`_score_kinds`](../../src/orchestrator/pkg/accuracy.py#L214), [`_sum_scores`](../../src/orchestrator/pkg/accuracy.py#L123)
- **Fields**: `emitted`, `expected`, `kind`, `matched`
- **Documented in**: `docs/specs/build-documents/PKG-ACC-2-build.md#7-files`

### `ParityReport`

[`src/orchestrator/pkg/accuracy.py:348`](../../src/orchestrator/pkg/accuracy.py#L348)

- **Called by** (1): [`score_parity`](../../src/orchestrator/pkg/accuracy.py#L382)
- **Fields**: `counts`

### `Regression`

[`src/orchestrator/pkg/accuracy.py:437`](../../src/orchestrator/pkg/accuracy.py#L437)

- **Called by** (1): [`compare_scoreboard`](../../src/orchestrator/pkg/accuracy.py#L570)
- **Fields**: `detail`, `metric`, `now`, `was`

## Functions

### `_case_batch`

[`src/orchestrator/pkg/accuracy.py:232`](../../src/orchestrator/pkg/accuracy.py#L232)

- **Called by** (1): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Calls** (4): [`RepoCodeExtractor`](../../src/orchestrator/pkg/extractor.py#L577), [`joins_from_list`](../../src/orchestrator/pkg/repos.py#L72), [`link_joins`](../../src/orchestrator/pkg/join_link.py#L150), [`merge_repos`](../../src/orchestrator/pkg/scoping.py#L150)

### `_describe`

[`src/orchestrator/pkg/accuracy.py:225`](../../src/orchestrator/pkg/accuracy.py#L225)

- **Called by** (1): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)

### `_edge_key`

[`src/orchestrator/pkg/accuracy.py:151`](../../src/orchestrator/pkg/accuracy.py#L151)

- **Called by** (2): [`_load_case`](../../src/orchestrator/pkg/accuracy.py#L160), [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Calls** (1): [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53)

### `_load_case`

[`src/orchestrator/pkg/accuracy.py:160`](../../src/orchestrator/pkg/accuracy.py#L160)

- **Called by** (2): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258), [`score_corpus`](../../src/orchestrator/pkg/accuracy.py#L307)
- **Calls** (4): [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53), [`_edge_key`](../../src/orchestrator/pkg/accuracy.py#L151), [`_require`](../../src/orchestrator/pkg/accuracy.py#L136), `loads`

### `_node_key`

[`src/orchestrator/pkg/accuracy.py:142`](../../src/orchestrator/pkg/accuracy.py#L142)

- **Called by** (1): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Calls** (1): [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53)

### `_ratio`

[`src/orchestrator/pkg/accuracy.py:449`](../../src/orchestrator/pkg/accuracy.py#L449)

- **Called by** (2): [`compare_scoreboard`](../../src/orchestrator/pkg/accuracy.py#L570), [`scoreboard_improvements`](../../src/orchestrator/pkg/accuracy.py#L627)
- **Calls** (1): `fractions.Fraction`

### `_require`

[`src/orchestrator/pkg/accuracy.py:136`](../../src/orchestrator/pkg/accuracy.py#L136)

- **Called by** (1): [`_load_case`](../../src/orchestrator/pkg/accuracy.py#L160)
- **Calls** (1): [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53)

### `_score_entry`

[`src/orchestrator/pkg/accuracy.py:454`](../../src/orchestrator/pkg/accuracy.py#L454)

- **Called by** (1): [`build_scoreboard`](../../src/orchestrator/pkg/accuracy.py#L458)

### `_score_kinds`

[`src/orchestrator/pkg/accuracy.py:214`](../../src/orchestrator/pkg/accuracy.py#L214)

- **Called by** (1): [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Calls** (1): [`KindScore`](../../src/orchestrator/pkg/accuracy.py#L58)

### `_sum_scores`

[`src/orchestrator/pkg/accuracy.py:123`](../../src/orchestrator/pkg/accuracy.py#L123)

- **Called by** (1): [`totals`](../../src/orchestrator/pkg/accuracy.py#L111)
- **Calls** (1): [`KindScore`](../../src/orchestrator/pkg/accuracy.py#L58)

### `build_scoreboard`

[`src/orchestrator/pkg/accuracy.py:458`](../../src/orchestrator/pkg/accuracy.py#L458)

- **Called by** (1 production · 3 test): [`_scoreboard`](../../src/orchestrator/cli.py#L3297), [`test_the_committed_baseline_matches_the_tree`](../../tests/pkg/test_scoreboard.py#L151), [`test_the_runtime_oracle_is_absent_unless_asked_for`](../../tests/pkg/test_scoreboard.py#L146), [`test_the_scoreboard_is_deterministic`](../../tests/pkg/test_scoreboard.py#L139)
- **Calls** (5): [`_score_entry`](../../src/orchestrator/pkg/accuracy.py#L454), [`score_corpus`](../../src/orchestrator/pkg/accuracy.py#L307), [`score_invention`](../../src/orchestrator/pkg/invention.py#L412), [`score_parity`](../../src/orchestrator/pkg/accuracy.py#L382), [`score_runtime`](../../src/orchestrator/pkg/runtime_oracle.py#L308)
- **Documented in**: `docs/specs/build-documents/PKG-ACC-5-build.md#7-files`

### `compare_scoreboard`

[`src/orchestrator/pkg/accuracy.py:570`](../../src/orchestrator/pkg/accuracy.py#L570)

- **Called by** (1 production · 15 test): [`_scoreboard`](../../src/orchestrator/cli.py#L3297), [`test_a_corpus_improvement_is_not_a_regression`](../../tests/pkg/test_scoreboard.py#L74), [`test_a_corpus_precision_drop_is_a_regression`](../../tests/pkg/test_scoreboard.py#L61), [`test_a_corpus_recall_drop_is_a_regression`](../../tests/pkg/test_scoreboard.py#L69), [`test_a_fabricated_edge_fails_the_gate`](../../tests/pkg/test_accuracy.py#L338), [`test_a_kind_disappearing_is_a_regression_not_silence`](../../tests/pkg/test_scoreboard.py#L80), [`test_an_identical_board_has_no_regressions`](../../tests/pkg/test_scoreboard.py#L130), [`test_an_unmeasured_language_is_not_gated`](../../tests/pkg/test_accuracy.py#L367), [`test_invention_is_gated_at_zero_per_language`](../../tests/pkg/test_scoreboard.py#L95), [`test_parity_shortfall_ratchets_one_way`](../../tests/pkg/test_scoreboard.py#L88), [`test_the_committed_baseline_matches_the_tree`](../../tests/pkg/test_scoreboard.py#L151), [`test_the_gate_ignores_a_language_the_current_run_could_not_measure`](../../tests/pkg/test_accuracy.py#L242), [`test_the_gate_is_absolute_not_relative_to_the_baseline`](../../tests/pkg/test_accuracy.py#L355), [`test_the_invention_gate_ignores_the_baseline_entirely`](../../tests/pkg/test_scoreboard.py#L107), [`test_the_invention_rate_may_still_move_freely`](../../tests/pkg/test_scoreboard.py#L112), [`test_undefined_scores_are_not_treated_as_drops`](../../tests/pkg/test_scoreboard.py#L161)
- **Calls** (2): [`Regression`](../../src/orchestrator/pkg/accuracy.py#L437), [`_ratio`](../../src/orchestrator/pkg/accuracy.py#L449)
- **Documented in**: `docs/specs/build-documents/PKG-ACC-5-build.md#7-files`

### `measured_recall`

[`src/orchestrator/pkg/accuracy.py:542`](../../src/orchestrator/pkg/accuracy.py#L542)

- **Called by** (1): [`_blast_prose`](../../src/orchestrator/sdlc/builddoc.py#L649)
- **Calls** (1): `loads`
- **Documented in**: `docs/specs/build-documents/PKG-ACC-6-build.md#7-files`

### `score_case`

[`src/orchestrator/pkg/accuracy.py:258`](../../src/orchestrator/pkg/accuracy.py#L258)

- **Called by** (1 production · 13 test): [`score_corpus`](../../src/orchestrator/pkg/accuracy.py#L307), [`test_a_fully_labelled_case_scores_one`](../../tests/pkg/test_accuracy.py#L68), [`test_a_kind_with_nothing_expected_scores_none_not_one`](../../tests/pkg/test_accuracy.py#L128), [`test_a_missing_or_unparseable_case_file_is_an_error`](../../tests/pkg/test_accuracy.py#L181), [`test_a_root_that_is_not_a_directory_is_an_error`](../../tests/pkg/test_accuracy.py#L192), [`test_an_edge_to_an_external_target_counts_against_precision`](../../tests/pkg/test_accuracy.py#L100), [`test_an_unfound_fact_costs_recall_not_precision`](../../tests/pkg/test_accuracy.py#L78), [`test_an_unknown_kind_is_an_error`](../../tests/pkg/test_accuracy.py#L175), [`test_an_unlabelled_emission_costs_precision_not_recall`](../../tests/pkg/test_accuracy.py#L89), [`test_annotating_a_gap_does_not_change_the_score`](../../tests/pkg/test_accuracy.py#L139), [`test_external_nodes_stay_out_of_the_node_ratio`](../../tests/pkg/test_accuracy.py#L117), [`test_false_positive_that_is_also_labelled_true_is_an_error`](../../tests/pkg/test_accuracy.py#L168), [`test_known_gap_naming_an_unlabelled_edge_is_an_error`](../../tests/pkg/test_accuracy.py#L159), [`test_provenance_is_checked_only_where_a_label_opts_in`](../../tests/pkg/test_accuracy.py#L201)
- **Calls** (7): [`CaseReport`](../../src/orchestrator/pkg/accuracy.py#L80), [`_case_batch`](../../src/orchestrator/pkg/accuracy.py#L232), [`_describe`](../../src/orchestrator/pkg/accuracy.py#L225), [`_edge_key`](../../src/orchestrator/pkg/accuracy.py#L151), [`_load_case`](../../src/orchestrator/pkg/accuracy.py#L160), [`_node_key`](../../src/orchestrator/pkg/accuracy.py#L142), [`_score_kinds`](../../src/orchestrator/pkg/accuracy.py#L214)

### `score_corpus`

[`src/orchestrator/pkg/accuracy.py:307`](../../src/orchestrator/pkg/accuracy.py#L307)

- **Called by** (2 production · 4 test): [`build_scoreboard`](../../src/orchestrator/pkg/accuracy.py#L458), [`pkg_accuracy`](../../src/orchestrator/cli.py#L3359), [`test_a_case_whose_front_end_is_missing_is_skipped_not_scored_zero`](../../tests/pkg/test_accuracy.py#L224), [`test_an_empty_corpus_is_an_error`](../../tests/pkg/test_accuracy.py#L273), [`test_corpus_walks_every_case_and_filters_by_language`](../../tests/pkg/test_accuracy.py#L214), [`test_totals_sum_across_cases`](../../tests/pkg/test_accuracy.py#L264)
- **Calls** (6): [`AccuracyReport`](../../src/orchestrator/pkg/accuracy.py#L96), [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53), [`_load_case`](../../src/orchestrator/pkg/accuracy.py#L160), [`default_extractors`](../../src/orchestrator/pkg/extractor.py#L530), `pathlib.Path`, [`score_case`](../../src/orchestrator/pkg/accuracy.py#L258)
- **Documented in**: `docs/specs/build-documents/PKG-ACC-1-build.md#5-blast-radius-impact-neighbourhood`, `docs/specs/build-documents/PKG-ACC-1-build.md#7-files`, `docs/specs/build-documents/PKG-ACC-2-build.md#5-blast-radius-impact-neighbourhood`, `docs/specs/build-documents/PKG-ACC-2-build.md#7-files`

### `score_parity`

[`src/orchestrator/pkg/accuracy.py:382`](../../src/orchestrator/pkg/accuracy.py#L382)

- **Called by** (2 production · 4 test): [`_parity_oracle`](../../src/orchestrator/cli.py#L3155), [`build_scoreboard`](../../src/orchestrator/pkg/accuracy.py#L458), [`test_parity_needs_no_corpus_and_no_tests`](../../tests/pkg/test_accuracy.py#L314), [`test_parity_on_a_missing_repo_is_an_error`](../../tests/pkg/test_accuracy.py#L324), [`test_parity_reports_shortfall_when_the_graph_misses_a_route`](../../tests/pkg/test_accuracy.py#L289), [`test_parity_separates_surplus_from_shortfall`](../../tests/pkg/test_accuracy.py#L301)
- **Calls** (5): [`CorpusError`](../../src/orchestrator/pkg/accuracy.py#L53), [`ParityReport`](../../src/orchestrator/pkg/accuracy.py#L348), [`RepoCodeExtractor`](../../src/orchestrator/pkg/extractor.py#L577), `pathlib.Path`, [`source_parity_counts`](../../src/orchestrator/pkg/verify.py#L355)
- **Documented in**: `docs/specs/build-documents/PKG-ACC-3-build.md#7-files`

### `scoreboard_improvements`

[`src/orchestrator/pkg/accuracy.py:627`](../../src/orchestrator/pkg/accuracy.py#L627)

- **Called by** (1 production · 4 test): [`_scoreboard`](../../src/orchestrator/cli.py#L3297), [`test_a_corpus_improvement_is_not_a_regression`](../../tests/pkg/test_scoreboard.py#L74), [`test_an_identical_board_has_no_regressions`](../../tests/pkg/test_scoreboard.py#L130), [`test_parity_shortfall_ratchets_one_way`](../../tests/pkg/test_scoreboard.py#L88), [`test_undefined_scores_are_not_treated_as_drops`](../../tests/pkg/test_scoreboard.py#L161)
- **Calls** (1): [`_ratio`](../../src/orchestrator/pkg/accuracy.py#L449)

## Imports

`__future__.annotations`, `dataclasses.dataclass`, `fractions.Fraction`, `json`, [`orchestrator.pkg.extractor`](../../src/orchestrator/pkg/extractor.py#L1), [`orchestrator.pkg.facts`](../../src/orchestrator/pkg/facts.py#L1), [`orchestrator.pkg.invention`](orchestrator.pkg.invention.md), [`orchestrator.pkg.join_link`](../../src/orchestrator/pkg/join_link.py#L1), [`orchestrator.pkg.repos`](../../src/orchestrator/pkg/repos.py#L1), [`orchestrator.pkg.runtime_oracle`](../../src/orchestrator/pkg/runtime_oracle.py#L1), [`orchestrator.pkg.scoping`](../../src/orchestrator/pkg/scoping.py#L1), [`orchestrator.pkg.verify`](orchestrator.pkg.verify.md), `pathlib.Path`, `typing.Any`

## Imported by

[`orchestrator.cli`](orchestrator.cli.md), [`orchestrator.sdlc.builddoc`](orchestrator.sdlc.builddoc.md), [`tests.pkg.test_accuracy`](../../tests/pkg/test_accuracy.py#L1), [`tests.pkg.test_scoreboard`](../../tests/pkg/test_scoreboard.py#L1)
