<USER_REQUEST>
Meridian augmentation init · PY

"""Phase 13 — Augmentation Triage. The first phase of Milestone 4.

 

Decides which of the (potentially thousands of) Findings from Phase 12

are worth sending to an LLM at all. Running Phase 15/16 against every

control — including every cleanly VIOLATED one with zero evidence — is

pure wasted cost and latency: there's nothing for an LLM to add when

the deterministic layer already has a clean, confident answer.

 

Two real, separately-labeled reasons a Finding can get selected — the

second one OFF by default, because checking real data showed the

obvious-looking default for it was nearly useless:

 

  1. AMBIGUOUS_EVIDENCE — verdict is PARTIAL or INCONCLUSIVE. Evidence

     exists but doesn't cleanly resolve. This is the textbook case

     "nuanced, non-deterministic judgment" was meant for. Enabled by

     default; on real Flask output this alone selects 593/1445 (41%)

     of controls — the genuine, meaningful reduction this phase exists

     to provide.

 

  2. HIGH_SEVERITY_GAP — verdict is VIOLATED (zero evidence found) AND

     the control's severity is in a configured set, empty by default.

     This is NOT a confidence signal — every VIOLATED finding has

     confidence=0.0 by construction (zero evidence always averages to

     exactly zero), so there's no "unusually low confidence VIOLATED"

     subset to carve out, which an earlier draft of this design

     assumed before checking real numbers. The real reason to flag

     these is different: a high-stakes control with literally no

     detected evidence is worth a second opinion on whether the

     deterministic rules have a real coverage gap. But checking real

     meridian.db data found that 1,326 of 1,445 controls (92%) are

     tagged Critical or High — a structural property of how

     regulatory severity scales work, not specific to any repository —

     so defaulting this filter to ("Critical", "High") selected 89% of

     VIOLATED findings on real Flask output, defeating the entire

     point of this phase. It's left configurable but OFF by default;

     turning it on is a deliberate cost/scope tradeoff for whoever

     operates Meridian to make explicitly, not a silent assumption.

 

Output is a TriageDecision per Finding, never a mutation of the

Finding itself — Phase 12's objects stay exactly as they were.

"""
Meridian augmentation models · PY

"""Phase 13 — TriageReason, TriageDecision."""

from __future__ import annotations

 

from dataclasses import dataclass

from enum import Enum

 

 

class TriageReason(str, Enum):

    AMBIGUOUS_EVIDENCE  = "ambiguous_evidence"   # PARTIAL or INCONCLUSIVE

    HIGH_SEVERITY_GAP   = "high_severity_gap"    # VIOLATED + Critical/High severity

 

 

@dataclass(frozen=True, slots=True)

class TriageDecision:

    finding_id:         str

    needs_llm_review:   bool

    reasons:            tuple[TriageReason, ...]   # empty iff needs_llm_review is False

 
Meridian augmentation triage · PY

"""Phase 13 — triage(). The one function this module exists for:

list[Finding] -> list[TriageDecision], one per Finding, deciding

whether it warrants LLM attention and exactly why.

 

Real finding from checking actual meridian.db data, not assumed:

across all 1,445 controls, 1,326 (92%) are tagged Critical or High

severity — that's a structural property of how regulatory severity

scales work (almost nothing gets marked "Low"), not specific to any

one repository. A first draft of this module defaulted

high_severity_gap_severities to ("Critical", "High") and, on real

Flask output, that alone selected 764 of 852 VIOLATED findings —

89%, which completely defeats the stated purpose of this phase

("prevents wasted LLM calls"). Even narrowing to Critical-only still

selects 195 zero-evidence findings by default.

 

Corrected default: high_severity_gap_severities is empty — this

reason is OFF unless explicitly configured. Sending a zero-evidence

Critical control to an LLM for a second opinion is a real, defensible

choice (catching coverage gaps in the deterministic rules), but it's

exactly the kind of cost/scope tradeoff that should be a deliberate

decision by whoever operates Meridian, not a silent default that

turns out to barely filter anything. AMBIGUOUS_EVIDENCE (PARTIAL /

INCONCLUSIVE) is the only reason enabled by default — on real Flask

output, that alone selects 593/1445 (41%), a genuine, meaningful

reduction from "review everything."

"""

from __future__ import annotations

 

from dataclasses import dataclass

 

from meridian.augmentation.models import TriageDecision, TriageReason

from meridian.verdicts.models import Finding, Verdict

 

_AMBIGUOUS_VERDICTS = (Verdict.PARTIAL, Verdict.INCONCLUSIVE)

 

 

@dataclass(frozen=True)

class TriageConfig:

    high_severity_gap_severities: tuple[str, ...] = ()   # OFF by default — see module docstring

 

 

def triage_findings(findings: list[Finding], config: TriageConfig | None = None) -> list[TriageDecision]:

    config = config or TriageConfig()

    decisions = []

    for f in findings:

        reasons: list[TriageReason] = []

 

        if f.verdict in _AMBIGUOUS_VERDICTS:

            reasons.append(TriageReason.AMBIGUOUS_EVIDENCE)

 

        if f.verdict == Verdict.VIOLATED and f.severity in config.high_severity_gap_severities:

            reasons.append(TriageReason.HIGH_SEVERITY_GAP)

 

        decisions.append(TriageDecision(

            finding_id=f.finding_id, needs_llm_review=bool(reasons), reasons=tuple(reasons),

        ))

    return decisions

 

 

def select_for_review(findings: list[Finding], decisions: list[TriageDecision]) -> list[Finding]:

    """Convenience filter — the actual Finding objects the triage marked

    for review, in the same order as the input list."""

    needed_ids = {d.finding_id for d in decisions if d.needs_llm_review}

    return [f for f in findings if f.finding_id in needed_ids]

Tests test augmentation models · PY

"""Phase 13 tests — TriageDecision, TriageReason."""

from __future__ import annotations

 

from meridian.augmentation.models import TriageDecision, TriageReason

 

 

def test_triage_reason_has_exactly_two_values() -> None:

    assert {r.value for r in TriageReason} == {"ambiguous_evidence", "high_severity_gap"}

 

 

def test_decision_not_needing_review_has_empty_reasons() -> None:

    d = TriageDecision(finding_id="f1", needs_llm_review=False, reasons=())

    assert d.reasons == ()

 

 

def test_decision_is_frozen() -> None:

    import pytest

    from dataclasses import FrozenInstanceError

    d = TriageDecision(finding_id="f1", needs_llm_review=False, reasons=())

    with pytest.raises(FrozenInstanceError):

        d.needs_llm_review = True  # type: ignore[misc]

 
Tests test augmentation triage · PY

"""Phase 13 tests — triage_findings() / select_for_review().

 

Includes a regression guard for the real design flaw found and fixed

while building this phase: defaulting HIGH_SEVERITY_GAP to

("Critical", "High") looked reasonable but, against real meridian.db

data, selected 89% of VIOLATED findings — because 92% of the entire

real ontology is tagged Critical or High severity, a structural

property of the data, not a meaningful filter. The fix was changing

the default to off (empty tuple), not adjusting the threshold value —

this test locks in that the DEFAULT must never silently re-introduce

the original problem.

"""

from __future__ import annotations

 

from meridian.augmentation.models import TriageReason

from meridian.augmentation.triage import TriageConfig, select_for_review, triage_findings

from meridian.verdicts.models import Finding, Verdict

 

 

def _finding(control_id: str, verdict: Verdict, severity: str | None = None) -> Finding:

    return Finding.create(

        framework_id="FW1", control_id=control_id, objective_ids=(), capability_ids=(),

        verdict=verdict, objective_coverage=0.0, confidence=0.9 if verdict != Verdict.VIOLATED else 0.0,

        evidence_ids=(), rationale="r", severity=severity, location_samples=(),

    )

 

 

# ── AMBIGUOUS_EVIDENCE ────────────────────────────────────────────────────────

 

def test_partial_verdict_triggers_ambiguous_evidence() -> None:

    f = _finding("C1", Verdict.PARTIAL, "High")

    decisions = triage_findings([f])

    assert decisions[0].needs_llm_review is True

    assert TriageReason.AMBIGUOUS_EVIDENCE in decisions[0].reasons

 

 

def test_inconclusive_verdict_triggers_ambiguous_evidence() -> None:

    f = _finding("C1", Verdict.INCONCLUSIVE, "Medium")

    decisions = triage_findings([f])

    assert TriageReason.AMBIGUOUS_EVIDENCE in decisions[0].reasons

 

 

def test_satisfied_verdict_never_triggers_review() -> None:

    f = _finding("C1", Verdict.SATISFIED, "Critical")

    decisions = triage_findings([f])

    assert decisions[0].needs_llm_review is False

    assert decisions[0].reasons == ()

 

 

def test_violated_verdict_does_not_trigger_ambiguous_evidence() -> None:

    f = _finding("C1", Verdict.VIOLATED, "Critical")

    decisions = triage_findings([f])

    assert TriageReason.AMBIGUOUS_EVIDENCE not in decisions[0].reasons

 

 

# ── HIGH_SEVERITY_GAP — off by default, the actual finding from this phase ───

 

def test_high_severity_gap_is_off_by_default_even_for_critical_violated() -> None:

    f = _finding("C1", Verdict.VIOLATED, "Critical")

    decisions = triage_findings([f])  # no config passed — default

    assert decisions[0].needs_llm_review is False

    assert decisions[0].reasons == ()

 

 

def test_default_config_selects_a_minority_not_a_majority_of_violated_findings() -> None:

    """Regression guard for the actual bug: with NO config override, a

    batch of VIOLATED findings across the real severity distribution

    must not be mostly selected — the whole point of the fix."""

    findings = (

        [_finding(f"C{i}", Verdict.VIOLATED, "High") for i in range(70)]

        + [_finding(f"D{i}", Verdict.VIOLATED, "Critical") for i in range(22)]

        + [_finding(f"E{i}", Verdict.VIOLATED, "Medium") for i in range(7)]

        + [_finding(f"F{i}", Verdict.VIOLATED, "Low") for i in range(1)]

    )  # mirrors the real ~70/22/7/1 percentage split found in meridian.db

    decisions = triage_findings(findings)

    selected_count = sum(1 for d in decisions if d.needs_llm_review)

    assert selected_count == 0  # default: zero VIOLATED findings selected, by design

 

 

def test_high_severity_gap_enabled_via_explicit_config() -> None:

    f = _finding("C1", Verdict.VIOLATED, "Critical")

    config = TriageConfig(high_severity_gap_severities=("Critical",))

    decisions = triage_findings([f], config)

    assert decisions[0].needs_llm_review is True

    assert TriageReason.HIGH_SEVERITY_GAP in decisions[0].reasons

 

 

def test_high_severity_gap_respects_configured_severity_set() -> None:

    config = TriageConfig(high_severity_gap_severities=("Critical",))

    high = _finding("C1", Verdict.VIOLATED, "High")

    critical = _finding("C2", Verdict.VIOLATED, "Critical")

    decisions = triage_findings([high, critical], config)

    assert decisions[0].needs_llm_review is False   # High not in configured set

    assert decisions[1].needs_llm_review is True    # Critical is

 

 

def test_high_severity_gap_does_not_apply_to_non_violated_verdicts() -> None:

    """Even with the gap reason enabled, it must only ever apply to

    VIOLATED — a PARTIAL Critical-severity finding should be flagged

    via AMBIGUOUS_EVIDENCE, never HIGH_SEVERITY_GAP."""

    config = TriageConfig(high_severity_gap_severities=("Critical", "High"))

    f = _finding("C1", Verdict.PARTIAL, "Critical")

    decisions = triage_findings([f], config)

    assert TriageReason.HIGH_SEVERITY_GAP not in decisions[0].reasons

    assert TriageReason.AMBIGUOUS_EVIDENCE in decisions[0].reasons

 

 

def test_finding_can_have_both_reasons_in_principle() -> None:

    """Structurally possible even if rare: not actually reachable given

    AMBIGUOUS_EVIDENCE requires PARTIAL/INCONCLUSIVE and

    HIGH_SEVERITY_GAP requires VIOLATED (mutually exclusive verdicts) —

    this test documents that exclusivity explicitly rather than leaving

    it implicit."""

    config = TriageConfig(high_severity_gap_severities=("Critical",))

    partial = _finding("C1", Verdict.PARTIAL, "Critical")

    violated = _finding("C2", Verdict.VIOLATED, "Critical")

    decisions = triage_findings([partial, violated], config)

    assert len(decisions[0].reasons) == 1

    assert len(decisions[1].reasons) == 1

 

 

# ── select_for_review ─────────────────────────────────────────────────────────

 

def test_select_for_review_returns_only_flagged_findings_in_order() -> None:

    f1 = _finding("C1", Verdict.SATISFIED, "High")

    f2 = _finding("C2", Verdict.PARTIAL, "High")

    f3 = _finding("C3", Verdict.VIOLATED, "Critical")

    decisions = triage_findings([f1, f2, f3])

    selected = select_for_review([f1, f2, f3], decisions)

    assert selected == [f2]

 

 

def test_select_for_review_empty_when_nothing_flagged() -> None:

    f1 = _finding("C1", Verdict.SATISFIED, "High")

    decisions = triage_findings([f1])

    assert select_for_review([f1], decisions) == []

Tests test augmentation extraction e2e · PY

"""Phase 13 — end-to-end against real Flask + real meridian.db. Locks

in the exact numbers verified while building this phase: 593/1445

(41.0%) selected by default, the real severity-distribution finding

that justified turning HIGH_SEVERITY_GAP off by default, and the

opt-in path's real impact.

"""

from __future__ import annotations

 

from pathlib import Path

 

import pytest

 

from meridian.augmentation.triage import TriageConfig, select_for_review, triage_findings

from meridian.compliance.resolver import ComplianceGraphResolver

from meridian.detectors.detector import detect_all_capabilities

from meridian.discovery import discover_repository

from meridian.evidence.engine import build_evidence_ledger

from meridian.facts.extractor import extract_all_facts

from meridian.file_ranker import rank_files

from meridian.graph.graph_builder import build_repository_graph

from meridian.inventory.aggregator import aggregate_inventory

from meridian.ir_builder import build_ir_repository

from meridian.parsers.parser_factory import parse_file

from meridian.sites.finder import discover_all_sites

from meridian.verdicts.models import Verdict

from meridian.verdicts.resolver import ControlVerdictResolver

 

FLASK_REPO = Path("/tmp/flask-repo")

DB_PATH = Path("/home/claude/meridian.db")

 

 

@pytest.fixture(scope="module")

def flask_findings():

    if not FLASK_REPO.exists():

        pytest.skip("Flask repo not present at /tmp/flask-repo in this sandbox session")

    if not DB_PATH.exists():

        pytest.skip("meridian.db not present in this sandbox session")

 

    md = discover_repository(FLASK_REPO)

    ranked = rank_files(md)

    parsed = [

        parse_file(rf.file.path, rf.file.rel_path, rf.file.language, rf.file.path.read_bytes())

        for rf in ranked

    ]

    repo = build_ir_repository(str(FLASK_REPO), parsed)

    graph = build_repository_graph(repo, repository_path=str(FLASK_REPO))

    facts = extract_all_facts(repo, graph)

    sites = discover_all_sites(repo, graph, facts)

    resolver = ComplianceGraphResolver(DB_PATH)

    claims = detect_all_capabilities(repo, graph, facts, sites, resolver)

    ledger = build_evidence_ledger(claims, facts)

    inventory = aggregate_inventory(ledger, sites, resolver, str(FLASK_REPO))

    return resolver, ControlVerdictResolver(resolver).resolve_all(inventory, ledger)

 

 

def test_real_severity_distribution_is_92_percent_high_or_critical(flask_findings) -> None:

    """The actual finding that justified this phase's design correction

    — locked in as a permanent regression check on the real DB, since

    if this ever changes substantially, the "off by default" decision

    should be revisited."""

    resolver, _ = flask_findings

    from collections import Counter

    counts = Counter(resolver.control(cid).severity for cid in resolver.all_control_ids())

    high_or_critical = counts.get("High", 0) + counts.get("Critical", 0)

    total = sum(counts.values())

    assert high_or_critical / total > 0.85  # confirms the real, structural skew

 

 

def test_default_triage_selects_exactly_the_593_ambiguous_findings(flask_findings) -> None:

    _, findings = flask_findings

    decisions = triage_findings(findings)

    selected = select_for_review(findings, decisions)

    assert len(selected) == 593

    assert all(f.verdict == Verdict.PARTIAL for f in selected)  # no INCONCLUSIVE exist on Flask

 

 

def test_default_triage_selection_fraction_is_meaningful_not_total(flask_findings) -> None:

    _, findings = flask_findings

    decisions = triage_findings(findings)

    selected = select_for_review(findings, decisions)

    fraction = len(selected) / len(findings)

    assert 0.3 < fraction < 0.5   # meaningfully less than "review everything"

 

 

def test_opt_in_critical_only_selects_more_but_remains_explicit(flask_findings) -> None:

    _, findings = flask_findings

    config = TriageConfig(high_severity_gap_severities=("Critical",))

    decisions = triage_findings(findings, config)

    selected = select_for_review(findings, decisions)

    assert len(selected) == 788  # verified manually: 593 ambiguous + 195 critical-violated

 

 

def test_no_satisfied_finding_is_ever_selected(flask_findings) -> None:

    """Flask has zero SATISFIED findings, so this is a structural

    assertion rather than a real-data one — included because it's the

    one verdict that should NEVER appear in triage output regardless."""

    _, findings = flask_findings

    decisions = triage_findings(findings)

    decision_by_id = {d.finding_id: d for d in decisions}

    for f in findings:

        if f.verdict == Verdict.SATISFIED:

            assert decision_by_id[f.finding_id].needs_llm_review is False

 

 

def test_triage_does_not_mutate_findings(flask_findings) -> None:

    _, findings = flask_findings

    before = [(f.verdict, f.confidence, f.severity) for f in findings]

    triage_findings(findings)

    after = [(f.verdict, f.confidence, f.severity) for f in findings]

    assert before == after

 



</USER_REQUEST>
<ADDITIONAL_METADATA>
The current local time is: 2026-06-25T18:57:54+05:30.

The user's current state is as follows:
Active Document: d:\meridian-mcp\src\meridian\evaluator.py (LANGUAGE_PYTHON)
Cursor is on line: 722
Other open documents:
- d:\meridian-mcp\src\meridian\evaluator.py (LANGUAGE_PYTHON)
- d:\meridian-mcp\src\meridian\evidence\engine.py (LANGUAGE_PYTHON)
- d:\meridian-mcp\src\meridian\discovery.py (LANGUAGE_PYTHON)
- d:\meridian-mcp\src\meridian\file_ranker.py (LANGUAGE_PYTHON)
- d:\meridian-mcp\src\meridian\ledger.py (LANGUAGE_PYTHON)
</ADDITIONAL_METADATA>