On the SYNERGY+ benchmark (v3), with the threshold frozen on 20 other reviews, records scoring below it in the 23 test reviews (33,001 records, 597 included studies) contained none of the studies finally included in any of them, while 29% of all records fell below it. Recalibrating with each of 43 reviews held out in turn kept ≥98% of included studies in 43 of 43 reviews and all of them in 42. Against human title/abstract decisions the threshold is stricter: it kept 96% of the records humans passed at that stage (range 78–100%, 12 reviews), and none of the records it lost was finally included. In 3 pre-specified external tests on 68 Cochrane reviews from CLEF TAR, the threshold kept 1,608 of 1,609 finally included studies; 1 of the 3 tests failed its pre-specified rule (at least 98% of the includes kept in every review), losing 1 study.
