A Nasute run, start to finish

A Nasute run, start to finish A workflow diagram generated by Archify. 01 / Set up 02 / Ask 03 / Judge 04 / Score and save 05 / Report EX / Errors Settings + profile · smoke or deep · Set up Settings + profile smoke or deep Load dataset · version + fingerprint · Set up Load dataset version + fingerprint Ask the target · each question x repeats · Ask Ask the target each question x repeats Answer · sources + extracts · Ask Answer sources + extracts The judge · yes or no per claim · Judge The judge yes or no per claim Judge error · retried, then left out · Judge · never held against it Judge error retried, then left out never held against it Count verdicts · into scores · Score and save Count verdicts into scores Save every row · results.jsonl · Score and save Save every row results.jsonl Summarise · causes, consistency · Score and save · means, clustered SE, 95% CI Summarise causes, consistency means, clustered SE, 95% CI Compare to baseline · gain, drop, noise or n/a · Score and save · paired verdict Compare to baseline gain, drop, noise or n/a paired verdict Report pages · dashboard, dataset, run, compare · Report Report pages dashboard, dataset, run, compare System error · scored 0, strict · Errors System error scored 0, strict reply next question verdicts fails left out no answer saved with 0s Legend Nasute code Error branch Saved file Judge model System under test