cohort A: 93 companies, 243 pairs, 82 scored; selection [668, 414, 330, 243]; hash 08605b3c80513bca; data 13dd7df7e326044a; from cas-schedule-p publish 20260613_041006
  benchmarks fitted in 75s; refusals {'mack': 0, 'mcl': 0}
cohort A tlrn_13_fh: data hash differs (fits recorded 819b8101576f90be, evaluator built 13dd7df7e326044a); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
cohort A tlrn_13_fh_factors: data hash differs (fits recorded 819b8101576f90be, evaluator built 13dd7df7e326044a); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
  benchmarks fitted in 79s; refusals {'mack': 0, 'mcl': 0, 'sur': 2}
=== cohort A: 93 companies, 243 pairs, 82 scored; selection [668, 414, 330, 243]; hash 08605b3c80513bca; data 13dd7df7e326044a; from cas-schedule-p publish 20260613_041006
status of any verdict on this cohort: confirmatory of a fixed specification, not out-of-sample; valuation 10 is 2007-12-31
scored companies: 82 of the cohort's board 82; cells scored at valuation 10: 9540
dominant company 1767 holds 66.3% of the actual valuation-10 reserve
data hash, tlrn_13_fh: cohort A tlrn_13_fh: data hash differs (fits recorded 819b8101576f90be, evaluator built 13dd7df7e326044a); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
data hash, tlrn_13_fh_factors: cohort A tlrn_13_fh_factors: data hash differs (fits recorded 819b8101576f90be, evaluator built 13dd7df7e326044a); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
first study accident-year rows: read from C:\Users\EthanKang\Projects\ibnr\analysis\results\tlrn_study\fits; tlrn_8_ay: 10 draws; tlrn_13_ay: 10 draws
notebook 04 Pool_APE at valuation 10: 
   mack ours 0.0577322; notebook_04 0.0577322; relative 6.66134e-15
   mcl ours 0.0566061; notebook_04 0.0566061; relative 2.34035e-13
first study benchmarks at valuation 10: 
   mack max_relative_difference 1.69326e-15
   mcl max_relative_difference 1.81435e-15
   actual max_relative_difference 0
benchmark Pool_APE by valuation [6, 7, 8, 10]:
   mack: 6.031%  4.363%  3.426%  5.773%  | rolling 4.898%
   mcl: 6.248%  4.383%  3.495%  5.661%  | rolling 4.947%

--- raw_network: 10 draws
   Pool_APE per draw at valuation 6: min 10.419%  median 10.980%  max 11.835%  sd 0.394%
   Pool_APE per draw at valuation 7: min 3.930%  median 4.399%  max 4.908%  sd 0.314%
   Pool_APE per draw at valuation 8: min 3.216%  median 4.325%  max 5.232%  sd 0.563%
   Pool_APE per draw at valuation 10: min 5.449%  median 5.850%  max 6.145%  sd 0.251%
   Pool_APE per draw at valuation rolling: min 5.967%  median 6.480%  max 6.645%  sd 0.223%

--- tlrn_13_fh: 10 draws
   Pool_APE per draw at valuation 6: min 8.517%  median 8.915%  max 9.123%  sd 0.189%
   Pool_APE per draw at valuation 7: min 3.818%  median 4.435%  max 4.655%  sd 0.273%
   Pool_APE per draw at valuation 8: min 3.116%  median 3.773%  max 4.183%  sd 0.300%
   Pool_APE per draw at valuation 10: min 4.929%  median 5.127%  max 5.423%  sd 0.168%
   Pool_APE per draw at valuation rolling: min 5.259%  median 5.532%  max 5.704%  sd 0.141%

--- tlrn_13_fh_factors: 10 draws
   Pool_APE per draw at valuation 6: min 7.462%  median 7.554%  max 7.669%  sd 0.070%
   Pool_APE per draw at valuation 7: min 4.259%  median 4.352%  max 4.455%  sd 0.072%
   Pool_APE per draw at valuation 8: min 4.315%  median 4.539%  max 4.769%  sd 0.133%
   Pool_APE per draw at valuation 10: min 6.708%  median 6.793%  max 6.827%  sd 0.038%
   Pool_APE per draw at valuation rolling: min 5.711%  median 5.805%  max 5.900%  sd 0.060%

Pool_APE of the draw average, and the other reported metrics:
            method valuation    n  pool_ape  pool_pe  medape         mae  log_ratio  n_log    mape        wrmse  share_better_than_mcl  share_better_than_mack
              mack         6 82.0   0.06031  0.02405 0.16491 13605.65326    0.29627   82.0 0.69975 545300.87669                0.31707                     NaN
              mack         7 82.0   0.04363  0.01923 0.13536  9574.34824    0.21332   82.0 0.23968 301216.35795                0.36585                     NaN
              mack         8 82.0   0.03426  0.01695 0.10661  6461.40604    0.19648   82.0 0.19956  41383.23921                0.48780                     NaN
              mack        10 82.0   0.05773  0.01030 0.13641 16150.95480    0.28552   81.0 1.15346 298622.49402                0.51220                     NaN
              mack   rolling  NaN   0.04898  0.01763 0.13582 11448.09058    0.24790    NaN 0.57311 296630.74197                0.42073                     NaN
               mcl         6 82.0   0.06248  0.02509 0.16423 14094.36939    0.30529   82.0 0.84561 545747.37301                    NaN                 0.30488
               mcl         7 82.0   0.04383  0.01987 0.12534  9618.15419    0.21691   82.0 0.24868 301267.48719                    NaN                 0.26829
               mcl         8 82.0   0.03495  0.01634 0.10797  6591.61466    0.24305   82.0 0.24805  41309.53097                    NaN                 0.42683
               mcl        10 82.0   0.05661 -0.00186 0.17827 15835.92379    0.27326   80.0 0.91794 240698.43172                    NaN                 0.48780
               mcl   rolling  NaN   0.04947  0.01486 0.14395 11535.01551    0.25963    NaN 0.56507 282255.70572                    NaN                 0.37195
       raw_network         6 82.0   0.10972  0.04983 0.21696 24749.99467    0.28961   82.0 0.58179 910908.56466                0.51220                 0.51220
       raw_network         7 82.0   0.04225  0.02136 0.16300  9271.84740    0.22009   82.0 0.25599 236809.13605                0.50000                 0.51220
       raw_network         8 82.0   0.04356 -0.01309 0.14549  8215.49030    0.18233   82.0 0.18738 265562.38724                0.47561                 0.42683
       raw_network        10 82.0   0.05802  0.03571 0.17741 16231.40512    0.24638   81.0 0.37287 167237.14313                0.48780                 0.47561
       raw_network   rolling  NaN   0.06339  0.02345 0.17571 14617.18437    0.23460    NaN 0.34951 395129.30777                0.49390                 0.48171
        tlrn_13_fh         6 82.0   0.08871  0.04367 0.16290 20010.15293    0.25274   82.0 0.63859 807277.90032                0.60976                 0.60976
        tlrn_13_fh         7 82.0   0.04285  0.02371 0.14654  9403.75321    0.20455   82.0 0.23848 277820.55526                0.51220                 0.48780
        tlrn_13_fh         8 82.0   0.03696 -0.00388 0.12559  6970.70216    0.17891   82.0 0.18606 187392.97797                0.53659                 0.43902
        tlrn_13_fh        10 82.0   0.05138  0.02150 0.14642 14372.57479    0.24014   81.0 0.53433 185748.96136                0.51220                 0.53659
        tlrn_13_fh   rolling  NaN   0.05497  0.02125 0.14536 12689.29577    0.21909    NaN 0.39936 364560.09873                0.54268                 0.51829
tlrn_13_fh_factors         6 82.0   0.07557 -0.00925 0.16368 17047.15466    0.28705   82.0 0.66373 369552.64109                0.51220                 0.52439
tlrn_13_fh_factors         7 82.0   0.04346 -0.01337 0.12597  9538.54612    0.21992   82.0 0.24667 104125.36741                0.43902                 0.48780
tlrn_13_fh_factors         8 82.0   0.04518 -0.01716 0.14203  8522.45243    0.22424   82.0 0.23164 211280.75004                0.48780                 0.37805
tlrn_13_fh_factors        10 82.0   0.06782 -0.02157 0.18297 18973.63604    0.27527   80.0 0.80361 381459.92968                0.52439                 0.51220
tlrn_13_fh_factors   rolling  NaN   0.05801 -0.01534 0.15366 13520.44731    0.25162    NaN 0.48641 266604.67206                0.49085                 0.47561

Difference in Pool_APE (row minus benchmark; negative favours the row), 95% interval:
            method benchmark    statistic     diff      low    high  share_favour
        tlrn_13_fh      mack         last -0.00636 -0.02166 0.02578        0.7730
        tlrn_13_fh      mack       pooled  0.00544 -0.00934 0.02071        0.1496
        tlrn_13_fh      mack      rolling  0.00599 -0.00982 0.01933        0.1650
        tlrn_13_fh      mack  valuation 6  0.02839  0.00706 0.07361           NaN
        tlrn_13_fh      mack  valuation 7 -0.00078 -0.01372 0.02373           NaN
        tlrn_13_fh      mack  valuation 8  0.00270 -0.05137 0.01675           NaN
        tlrn_13_fh      mack valuation 10 -0.00636 -0.02166 0.02578           NaN
        tlrn_13_fh       mcl         last -0.00523 -0.05143 0.02136        0.7394
        tlrn_13_fh       mcl       pooled  0.00505 -0.01667 0.01342        0.2138
        tlrn_13_fh       mcl      rolling  0.00551 -0.01532 0.01318        0.2270
        tlrn_13_fh       mcl  valuation 6  0.02622 -0.00144 0.07144           NaN
        tlrn_13_fh       mcl  valuation 7 -0.00098 -0.01204 0.02311           NaN
        tlrn_13_fh       mcl  valuation 8  0.00201 -0.05327 0.01607           NaN
        tlrn_13_fh       mcl valuation 10 -0.00523 -0.05143 0.02136           NaN
tlrn_13_fh_factors      mack         last  0.01009 -0.03829 0.09367        0.2461
tlrn_13_fh_factors      mack       pooled  0.00908 -0.00021 0.06731        0.0325
tlrn_13_fh_factors      mack      rolling  0.00903  0.00022 0.06229        0.0193
tlrn_13_fh_factors      mack  valuation 6  0.01526 -0.01059 0.11628           NaN
tlrn_13_fh_factors      mack  valuation 7 -0.00016 -0.01576 0.05364           NaN
tlrn_13_fh_factors      mack  valuation 8  0.01093 -0.03089 0.02362           NaN
tlrn_13_fh_factors      mack valuation 10  0.01009 -0.03829 0.09367           NaN
tlrn_13_fh_factors       mcl         last  0.01122 -0.02189 0.05264        0.1559
tlrn_13_fh_factors       mcl       pooled  0.00869  0.00016 0.05458        0.0214
tlrn_13_fh_factors       mcl      rolling  0.00854  0.00031 0.05246        0.0188
tlrn_13_fh_factors       mcl  valuation 6  0.01309 -0.01107 0.11266           NaN
tlrn_13_fh_factors       mcl  valuation 7 -0.00036 -0.01581 0.05535           NaN
tlrn_13_fh_factors       mcl  valuation 8  0.01024 -0.03000 0.02165           NaN
tlrn_13_fh_factors       mcl valuation 10  0.01122 -0.02189 0.05264           NaN
       raw_network      mack         last  0.00029 -0.01258 0.06031        0.4527
       raw_network      mack       pooled  0.01388  0.00497 0.05278        0.0057
       raw_network      mack      rolling  0.01440  0.00400 0.05002        0.0094
       raw_network      mack  valuation 6  0.04940  0.02904 0.15590           NaN
       raw_network      mack  valuation 7 -0.00138 -0.00845 0.03741           NaN
       raw_network      mack  valuation 8  0.00930 -0.05703 0.02520           NaN
       raw_network      mack valuation 10  0.00029 -0.01258 0.06031           NaN
       raw_network       mcl         last  0.00141 -0.02142 0.05038        0.4471
       raw_network       mcl       pooled  0.01350  0.00339 0.04349        0.0148
       raw_network       mcl      rolling  0.01392  0.00169 0.04211        0.0185
       raw_network       mcl  valuation 6  0.04724  0.02075 0.15362           NaN
       raw_network       mcl  valuation 7 -0.00158 -0.00842 0.03684           NaN
       raw_network       mcl  valuation 8  0.00861 -0.05892 0.02452           NaN
       raw_network       mcl valuation 10  0.00141 -0.02142 0.05038           NaN

Valuation-10 difference without the dominant company:
            method benchmark     diff      low    high  share_favour
        tlrn_13_fh      mack  0.00093 -0.03213 0.03389        0.4876
        tlrn_13_fh       mcl -0.00257 -0.07237 0.02720        0.5599
tlrn_13_fh_factors      mack  0.01277 -0.04917 0.11384        0.3685
tlrn_13_fh_factors       mcl  0.00927 -0.02820 0.06867        0.3647
       raw_network      mack  0.02837 -0.00899 0.06929        0.0533
       raw_network       mcl  0.02486 -0.04275 0.05887        0.1801

Pool_APE within quartiles of the actual reserve:
method                    mack      mcl  raw_network  tlrn_13_fh  tlrn_13_fh_factors
valuation quartile n                                                                
6         1        21  0.64763  0.67420      0.50418     0.50552             0.52918
          2        20  0.23344  0.31841      0.20021     0.20231             0.25053
          3        21  0.16733  0.16094      0.23270     0.17368             0.17695
          4        20  0.05372  0.05520      0.10390     0.08383             0.06941
7         1        21  0.35331  0.50383      0.37006     0.36938             0.45150
          2        20  0.19322  0.17287      0.22278     0.19341             0.16402
          3        21  0.19616  0.21583      0.18122     0.17267             0.17752
          4        20  0.03605  0.03547      0.03477     0.03602             0.03663
8         1        21  0.23402  0.33626      0.26997     0.26636             0.30682
          2        20  0.13404  0.19111      0.17268     0.15529             0.18006
          3        21  0.14193  0.17600      0.16353     0.16304             0.16796
          4        20  0.02898  0.02766      0.03747     0.03077             0.03886
10        1        21  2.31071  1.86696      0.58842     1.03751             1.58810
          2        20  0.24399  0.24201      0.23921     0.21576             0.24355
          3        21  0.18690  0.24627      0.16782     0.14056             0.20189
          4        20  0.04728  0.04450      0.05105     0.04467             0.05843

Precision (Amendment 2): the width of each 95% interval and the gain it could just detect, about half the width, in Pool_APE (0.03 is three points). A gain of half a point (0.005) is far below these, so 'not distinguishable' is the expected result on one cohort:
 bootstrap          benchmark statistic  width  detectable_gain
   uniform               mack      last 0.0474           0.0237
   uniform               mack    pooled 0.0301           0.0150
   uniform               mack   rolling 0.0291           0.0146
   uniform                mcl      last 0.0728           0.0364
   uniform                mcl    pooled 0.0301           0.0150
   uniform                mcl   rolling 0.0285           0.0142
stratified               mack      last 0.0191           0.0096
stratified               mack    pooled 0.0121           0.0061
stratified               mack   rolling 0.0123           0.0061
stratified                mcl      last 0.0286           0.0143
stratified                mcl    pooled 0.0113           0.0057
stratified                mcl   rolling 0.0112           0.0056
stratified tlrn_13_fh_factors      last 0.0463           0.0232
stratified tlrn_13_fh_factors    pooled 0.0168           0.0084
stratified tlrn_13_fh_factors   rolling 0.0164           0.0082

Sensitivity analysis, a stratified bootstrap (the dominant company in every resample once; it conditions on that company being present; not the verdict):
    method          benchmark statistic     diff      low    high  share_favour
tlrn_13_fh               mack      last -0.00636 -0.01564 0.00347        0.8936
tlrn_13_fh               mack    pooled  0.00544 -0.00013 0.01200        0.0286
tlrn_13_fh               mack   rolling  0.00599  0.00033 0.01262        0.0186
tlrn_13_fh                mcl      last -0.00523 -0.02139 0.00724        0.7835
tlrn_13_fh                mcl    pooled  0.00505 -0.00022 0.01110        0.0307
tlrn_13_fh                mcl   rolling  0.00551  0.00044 0.01161        0.0159
tlrn_13_fh tlrn_13_fh_factors      last -0.01645 -0.04020 0.00612        0.9227
tlrn_13_fh tlrn_13_fh_factors    pooled -0.00364 -0.01294 0.00381        0.7795
tlrn_13_fh tlrn_13_fh_factors   rolling -0.00304 -0.01169 0.00470        0.7288
   the same rule on the stratified intervals gives: NOT DISTINGUISHABLE

Comparison rows (row minus the main row; positive means the main row is better):
                   row    against statistic    diff      low    high  share_favour
           raw_network tlrn_13_fh      last 0.00664 -0.00102 0.04186        0.0648
           raw_network tlrn_13_fh    pooled 0.00844  0.00411 0.03363        0.0000
           raw_network tlrn_13_fh   rolling 0.00841  0.00447 0.03214        0.0000
    tlrn_13_fh_factors tlrn_13_fh      last 0.01645 -0.02902 0.08303        0.1540
    tlrn_13_fh_factors tlrn_13_fh    pooled 0.00364 -0.00578 0.05418        0.2999
    tlrn_13_fh_factors tlrn_13_fh   rolling 0.00304 -0.00678 0.05149        0.3323
 first_study:tlrn_8_ay tlrn_13_fh      last 0.00627 -0.03665 0.04635        0.3022
first_study:tlrn_13_ay tlrn_13_fh      last 0.01098 -0.04411 0.02758        0.3292

Median blend weight alpha over members and draws:
 valuation  alpha_median
         6        0.6060
         7        0.8004
         8        0.7655
        10        0.6317

Spread of the blend weight alpha over members and draws:
 valuation  alpha_min  alpha_q25  alpha_median  alpha_q75  alpha_max  alpha_std  alpha_count
         6     0.2766     0.4973        0.6060     0.7165     1.0000     0.1678          200
         7     0.3948     0.6896        0.8004     0.9207     1.0000     0.1542          200
         8     0.4301     0.6667        0.7655     0.8512     1.0000     0.1341          200
        10     0.4013     0.5431        0.6317     0.7149     0.9099     0.1096          200

VERDICT for tlrn_13_fh on cohort A (confirmatory of a fixed specification, not out-of-sample): NOT DISTINGUISHABLE
   vs mack: valuation 10 [-2.1657%, +2.5779%], pooled [-0.9340%, +2.0711%], valuation 10 without the dominant company +0.0932% [-3.2128%, +3.3891%]; vs mcl: valuation 10 [-5.1431%, +2.1364%], pooled [-1.6669%, +1.3422%], valuation 10 without the dominant company -0.2575% [-7.2369%, +2.7197%]; median-draw valuation-10 Pool_APE 5.1273% against mack 5.7732% and mcl 5.6606%; conditions i_last_below_both no, ii_pooled_below_both no, iii_median_draw_below_both yes, iv_excluding_dominant_not_worse no

Amendment 1, a separate question (not part of the verdict above): does the network add anything beyond the blend? Pool_APE difference, model minus the factors-only row, negative favours the model:
      model      compared_with statistic     diff      low     high  share_favour
 tlrn_13_fh tlrn_13_fh_factors      last -0.01645 -0.08303  0.02902        0.8460
 tlrn_13_fh tlrn_13_fh_factors   rolling -0.00304 -0.05149  0.00678        0.6677
 tlrn_13_fh tlrn_13_fh_factors    pooled -0.00364 -0.05418  0.00578        0.7001
 tlrn_13_fh tlrn_13_fh_factors excluding -0.01184 -0.10277  0.03845        0.6496
raw_network tlrn_13_fh_factors      last -0.00980 -0.05491  0.05744        0.6751
raw_network tlrn_13_fh_factors   rolling  0.00538 -0.02707  0.01280        0.3359
raw_network tlrn_13_fh_factors excluding  0.01559 -0.07916  0.06756        0.3698
 tlrn_13_fh        raw_network      last -0.00664 -0.04186  0.00102        0.9352
 tlrn_13_fh        raw_network   rolling -0.00841 -0.03214 -0.00447        1.0000
 tlrn_13_fh        raw_network    pooled -0.00844 -0.03363 -0.00411        1.0000
 tlrn_13_fh        raw_network excluding -0.02744 -0.04795 -0.01363        0.9994
NETWORK QUESTION for tlrn_13_fh on cohort A: NOT DISTINGUISHABLE
   valuation 10 [-8.3031%, +2.9022%]; rolling [-5.1493%, +0.6781%]; valuation 10 without the dominant company [-10.2767%, +3.8454%]; conditions a_last_below no, b_rolling_below no, c_excluding_not_above yes

Amendment 2, another question (outside the verdict): main row minus the raw network, by the same interval rules (negative favours the main row, i.e. the blend adds value):
BLEND QUESTION for tlrn_13_fh on cohort A: NOT DISTINGUISHABLE
   valuation 10 [-4.1858%, +0.1024%]; rolling [-3.2144%, -0.4468%]; valuation 10 without the dominant company [-4.7952%, -1.3634%]; conditions a_last_below no, b_rolling_below yes, c_excluding_not_above yes
(cohort A evaluated in 184s)
cohort B: 84 companies, 212 pairs, 84 scored; selection [668, 414, 297, 212]; hash de51bae56cca563c; data 2d9ecc777b088fcd; from cas-schedule-p publish 20260613_041006
  benchmarks fitted in 36s; refusals {'mack': 0, 'mcl': 0}
cohort B tlrn_13_fh: data hash differs (fits recorded bce499efbb6bde4e, evaluator built 2d9ecc777b088fcd); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
cohort B tlrn_13_fh_factors: data hash differs (fits recorded bce499efbb6bde4e, evaluator built 2d9ecc777b088fcd); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
  benchmarks fitted in 81s; refusals {'mack': 0, 'mcl': 0, 'sur': 0}
=== cohort B: 84 companies, 212 pairs, 84 scored; selection [668, 414, 297, 212]; hash de51bae56cca563c; data 2d9ecc777b088fcd; from cas-schedule-p publish 20260613_041006
status of any verdict on this cohort: confirmatory of a fixed specification, not out-of-sample; valuation 10 is 2007-12-31
scored companies: 84 of the cohort's board 84; cells scored at valuation 10: 9540
dominant company 1767 holds 67.3% of the actual valuation-10 reserve
data hash, tlrn_13_fh: cohort B tlrn_13_fh: data hash differs (fits recorded bce499efbb6bde4e, evaluator built 2d9ecc777b088fcd); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
data hash, tlrn_13_fh_factors: cohort B tlrn_13_fh_factors: data hash differs (fits recorded bce499efbb6bde4e, evaluator built 2d9ecc777b088fcd); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
first study accident-year rows: read from C:\Users\EthanKang\Projects\ibnr\analysis\results\tlrn_study\fits; tlrn_8_ay: 10 draws; tlrn_13_ay: 10 draws
first study benchmarks at valuation 10: 
   mack max_relative_difference 2.63746e-15
   mcl max_relative_difference 1.61203e-15
   actual max_relative_difference 0
companion's rolling table: 
 valuation  companies_ours  companies_theirs  cells_ours  cells_theirs  mack_ours  mack_theirs  mack_difference  mcl_ours  mcl_theirs  mcl_difference  tlrn_13_fh_draw_average  tlrn_13_fh_theirs  raw_network_draw_average  raw_network_theirs  alpha_median_ours  alpha_median_theirs
         6              84                84        2968          2968   0.060115     0.060115              0.0  0.061063    0.061063            -0.0                 0.091942           0.082403                  0.114077            0.095151           0.608919             0.527248
         7              84                84        3180          3180   0.044389     0.044389              0.0  0.044095    0.044095             0.0                 0.039541           0.035022                  0.039610            0.033551           0.813492             0.701999
         8              84                84        2756          2756   0.034834     0.034834              0.0  0.036108    0.036108             0.0                 0.033640           0.035769                  0.043621            0.039395           0.712751             0.735993
        10              84                84        9540          9540   0.054810     0.054810             -0.0  0.054098    0.054098            -0.0                 0.049624           0.049687                  0.055869            0.055561           0.607949             0.626732
members against the companion's published table (report only): 
 valuation  members_ours  members_theirs  alpha_median_ours  alpha_median_theirs  best_epoch_median_ours  best_epoch_median_theirs
         6           200              20             0.6089               0.5272                   130.0                     107.5
         7           200              20             0.8135               0.7020                   567.5                     370.0
         8           200              20             0.7128               0.7360                  1045.0                     930.0
        10           200              20             0.6079               0.6267                  1775.0                    1880.0
benchmark Pool_APE by valuation [6, 7, 8, 10]:
   mack: 6.012%  4.439%  3.483%  5.481%  | rolling 4.854%
   mcl: 6.106%  4.410%  3.611%  5.410%  | rolling 4.884%

--- raw_network: 10 draws
   Pool_APE per draw at valuation 6: min 10.554%  median 11.394%  max 12.577%  sd 0.679%
   Pool_APE per draw at valuation 7: min 3.368%  median 4.002%  max 4.540%  sd 0.394%
   Pool_APE per draw at valuation 8: min 3.542%  median 4.316%  max 5.399%  sd 0.685%
   Pool_APE per draw at valuation 10: min 5.169%  median 5.617%  max 5.938%  sd 0.226%
   Pool_APE per draw at valuation rolling: min 6.021%  median 6.384%  max 6.689%  sd 0.213%

--- tlrn_13_fh: 10 draws
   Pool_APE per draw at valuation 6: min 8.725%  median 9.230%  max 9.914%  sd 0.403%
   Pool_APE per draw at valuation 7: min 3.470%  median 3.947%  max 4.517%  sd 0.282%
   Pool_APE per draw at valuation 8: min 3.020%  median 3.346%  max 3.895%  sd 0.271%
   Pool_APE per draw at valuation 10: min 4.743%  median 5.013%  max 5.140%  sd 0.129%
   Pool_APE per draw at valuation rolling: min 5.246%  median 5.363%  max 5.548%  sd 0.100%

--- tlrn_13_fh_factors: 10 draws
   Pool_APE per draw at valuation 6: min 7.398%  median 7.570%  max 7.749%  sd 0.115%
   Pool_APE per draw at valuation 7: min 4.242%  median 4.409%  max 4.464%  sd 0.066%
   Pool_APE per draw at valuation 8: min 4.394%  median 4.654%  max 4.862%  sd 0.129%
   Pool_APE per draw at valuation 10: min 6.635%  median 6.660%  max 6.738%  sd 0.042%
   Pool_APE per draw at valuation rolling: min 5.759%  median 5.815%  max 5.873%  sd 0.038%

Pool_APE of the draw average, and the other reported metrics:
            method valuation    n  pool_ape  pool_pe  medape         mae  log_ratio  n_log    mape        wrmse  share_better_than_mcl  share_better_than_mack
              mack         6 84.0   0.06012  0.02501 0.16491 13052.28927    0.30021   84.0 0.68746 5.489082e+05                0.30952                     NaN
              mack         7 84.0   0.04439  0.01906 0.13623  9374.09744    0.22186   84.0 0.24653 3.032335e+05                0.41667                     NaN
              mack         8 84.0   0.03483  0.01756 0.10661  6327.18506    0.19461   84.0 0.19908 4.160367e+04                0.50000                     NaN
              mack        10 84.0   0.05481  0.00730 0.13417 14753.78105    0.28893   83.0 1.14061 3.003078e+05                0.53571                     NaN
              mack   rolling  NaN   0.04854  0.01723 0.13548 10876.83820    0.25140    NaN 0.56842 2.985133e+05                0.44048                     NaN
               mcl         6 84.0   0.06106  0.02535 0.16423 13258.05254    0.29752   84.0 0.80567 5.493479e+05                    NaN                 0.34524
               mcl         7 84.0   0.04410  0.02055 0.13739  9312.00147    0.23185   84.0 0.26437 3.032485e+05                    NaN                 0.23810
               mcl         8 84.0   0.03611  0.01680 0.10797  6558.70584    0.23655   84.0 0.24691 4.155153e+04                    NaN                 0.42857
               mcl        10 84.0   0.05410 -0.00468 0.17827 14562.20394    0.27767   82.0 0.90950 2.420255e+05                    NaN                 0.46429
               mcl   rolling  NaN   0.04884  0.01450 0.14696 10922.74095    0.26090    NaN 0.55661 2.840433e+05                    NaN                 0.36905
       raw_network         6 84.0   0.11408  0.06753 0.21024 24768.47129    0.28479   84.0 0.59908 1.029234e+06                0.46429                 0.45238
       raw_network         7 84.0   0.03961  0.00827 0.17761  8364.76113    0.22765   84.0 0.25639 1.670293e+05                0.45238                 0.46429
       raw_network         8 84.0   0.04362 -0.01477 0.14562  7923.26431    0.18072   84.0 0.18205 2.457917e+05                0.48810                 0.46429
       raw_network        10 84.0   0.05587  0.04018 0.18284 15039.02372    0.24659   83.0 0.36756 1.584780e+05                0.48810                 0.46429
       raw_network   rolling  NaN   0.06329  0.02530 0.17908 14023.88011    0.23493    NaN 0.35127 4.001333e+05                0.47321                 0.46131
        tlrn_13_fh         6 84.0   0.09194  0.05530 0.16783 19962.48937    0.24732   84.0 0.63290 8.893507e+05                0.60714                 0.63095
        tlrn_13_fh         7 84.0   0.03954  0.01466 0.16347  8350.20813    0.21183   84.0 0.24087 2.365854e+05                0.50000                 0.48810
        tlrn_13_fh         8 84.0   0.03364 -0.00191 0.13200  6110.29882    0.17927   84.0 0.18546 1.464486e+05                0.61905                 0.52381
        tlrn_13_fh        10 84.0   0.04962  0.02253 0.13278 13357.85702    0.23912   83.0 0.53518 1.809810e+05                0.53571                 0.55952
        tlrn_13_fh   rolling  NaN   0.05369  0.02264 0.14902 11945.21334    0.21939    NaN 0.39860 3.633414e+05                0.56548                 0.55060
tlrn_13_fh_factors         6 84.0   0.07550 -0.01158 0.16282 16392.13774    0.28575   84.0 0.64638 3.525566e+05                0.48810                 0.53571
tlrn_13_fh_factors         7 84.0   0.04383 -0.01278 0.14492  9256.83405    0.23431   84.0 0.26135 1.080220e+05                0.48810                 0.50000
tlrn_13_fh_factors         8 84.0   0.04650 -0.01805 0.13967  8446.11501    0.21853   84.0 0.22926 2.133229e+05                0.47619                 0.36905
tlrn_13_fh_factors        10 84.0   0.06676 -0.02597 0.19608 17970.94844    0.28011   82.0 0.79706 3.939915e+05                0.52381                 0.48810
tlrn_13_fh_factors   rolling  NaN   0.05815 -0.01710 0.16087 13016.50881    0.25467    NaN 0.48351 2.669733e+05                0.49405                 0.47321

Difference in Pool_APE (row minus benchmark; negative favours the row), 95% interval:
            method benchmark    statistic     diff      low    high  share_favour
        tlrn_13_fh      mack         last -0.00519 -0.01873 0.03505        0.7069
        tlrn_13_fh      mack       pooled  0.00486 -0.01072 0.02129        0.1775
        tlrn_13_fh      mack      rolling  0.00515 -0.01194 0.01948        0.2003
        tlrn_13_fh      mack  valuation 6  0.03183 -0.00080 0.07618           NaN
        tlrn_13_fh      mack  valuation 7 -0.00485 -0.01927 0.01917           NaN
        tlrn_13_fh      mack  valuation 8 -0.00119 -0.05279 0.01225           NaN
        tlrn_13_fh      mack valuation 10 -0.00519 -0.01873 0.03505           NaN
        tlrn_13_fh       mcl         last -0.00447 -0.04858 0.02860        0.6964
        tlrn_13_fh       mcl       pooled  0.00465 -0.01689 0.01458        0.2248
        tlrn_13_fh       mcl      rolling  0.00485 -0.01626 0.01383        0.2458
        tlrn_13_fh       mcl  valuation 6  0.03088 -0.00570 0.07983           NaN
        tlrn_13_fh       mcl  valuation 7 -0.00455 -0.01584 0.02046           NaN
        tlrn_13_fh       mcl  valuation 8 -0.00247 -0.05691 0.01110           NaN
        tlrn_13_fh       mcl valuation 10 -0.00447 -0.04858 0.02860           NaN
tlrn_13_fh_factors      mack         last  0.01195 -0.03724 0.10035        0.2196
tlrn_13_fh_factors      mack       pooled  0.00974 -0.00004 0.07328        0.0268
tlrn_13_fh_factors      mack      rolling  0.00961  0.00029 0.06782        0.0184
tlrn_13_fh_factors      mack  valuation 6  0.01538 -0.01222 0.12804           NaN
tlrn_13_fh_factors      mack  valuation 7 -0.00056 -0.01562 0.05447           NaN
tlrn_13_fh_factors      mack  valuation 8  0.01167 -0.03180 0.02577           NaN
tlrn_13_fh_factors      mack valuation 10  0.01195 -0.03724 0.10035           NaN
tlrn_13_fh_factors       mcl         last  0.01266 -0.02009 0.05916        0.1421
tlrn_13_fh_factors       mcl       pooled  0.00953  0.00038 0.06185        0.0179
tlrn_13_fh_factors       mcl      rolling  0.00931  0.00038 0.05967        0.0169
tlrn_13_fh_factors       mcl  valuation 6  0.01443 -0.01203 0.13027           NaN
tlrn_13_fh_factors       mcl  valuation 7 -0.00026 -0.01554 0.05836           NaN
tlrn_13_fh_factors       mcl  valuation 8  0.01039 -0.03166 0.02183           NaN
tlrn_13_fh_factors       mcl valuation 10  0.01266 -0.02009 0.05916           NaN
       raw_network      mack         last  0.00106 -0.01363 0.07517        0.4376
       raw_network      mack       pooled  0.01432  0.00636 0.05549        0.0039
       raw_network      mack      rolling  0.01476  0.00561 0.05171        0.0056
       raw_network      mack  valuation 6  0.05396  0.01881 0.15797           NaN
       raw_network      mack  valuation 7 -0.00478 -0.01330 0.03630           NaN
       raw_network      mack  valuation 8  0.00879 -0.05864 0.02279           NaN
       raw_network      mack valuation 10  0.00106 -0.01363 0.07517           NaN
       raw_network       mcl         last  0.00177 -0.02546 0.06659        0.4417
       raw_network       mcl       pooled  0.01411  0.00451 0.04706        0.0130
       raw_network       mcl      rolling  0.01445  0.00343 0.04523        0.0145
       raw_network       mcl  valuation 6  0.05301  0.01275 0.16126           NaN
       raw_network       mcl  valuation 7 -0.00449 -0.01271 0.03803           NaN
       raw_network       mcl  valuation 8  0.00751 -0.06395 0.02172           NaN
       raw_network       mcl valuation 10  0.00177 -0.02546 0.06659           NaN

Valuation-10 difference without the dominant company:
            method benchmark    diff      low    high  share_favour
        tlrn_13_fh      mack 0.00611 -0.02695 0.04406        0.3613
        tlrn_13_fh       mcl 0.00113 -0.07021 0.03549        0.4936
tlrn_13_fh_factors      mack 0.01692 -0.04823 0.12036        0.3367
tlrn_13_fh_factors       mcl 0.01195 -0.02732 0.07270        0.3276
       raw_network      mack 0.03552 -0.00568 0.08984        0.0424
       raw_network       mcl 0.03054 -0.04450 0.07756        0.1676

Pool_APE within quartiles of the actual reserve:
method                    mack      mcl  raw_network  tlrn_13_fh  tlrn_13_fh_factors
valuation quartile n                                                                
6         1        21  0.66979  0.69691      0.52541     0.53939             0.55573
          2        21  0.22146  0.21735      0.20970     0.14411             0.18658
          3        21  0.17580  0.17678      0.22396     0.17918             0.18448
          4        21  0.05333  0.05428      0.10863     0.08766             0.06976
7         1        21  0.37731  0.43629      0.38605     0.36381             0.42220
          2        21  0.16918  0.18545      0.21881     0.19410             0.16664
          3        21  0.16343  0.17891      0.17458     0.15680             0.15641
          4        21  0.03817  0.03702      0.03219     0.03306             0.03777
8         1        21  0.25112  0.34184      0.25527     0.25439             0.31413
          2        21  0.11837  0.18289      0.17426     0.16079             0.18443
          3        21  0.15510  0.17765      0.16519     0.16277             0.15323
          4        21  0.02931  0.02893      0.03750     0.02729             0.04069
10        1        21  2.43080  1.97595      0.57394     1.10465             1.68495
          2        21  0.25229  0.24946      0.23184     0.20150             0.24042
          3        21  0.20234  0.27421      0.19145     0.15405             0.22216
          4        21  0.04370  0.04104      0.04816     0.04254             0.05675

Precision (Amendment 2): the width of each 95% interval and the gain it could just detect, about half the width, in Pool_APE (0.03 is three points). A gain of half a point (0.005) is far below these, so 'not distinguishable' is the expected result on one cohort:
 bootstrap          benchmark statistic  width  detectable_gain
   uniform               mack      last 0.0538           0.0269
   uniform               mack    pooled 0.0320           0.0160
   uniform               mack   rolling 0.0314           0.0157
   uniform                mcl      last 0.0772           0.0386
   uniform                mcl    pooled 0.0315           0.0157
   uniform                mcl   rolling 0.0301           0.0150
stratified               mack      last 0.0201           0.0101
stratified               mack    pooled 0.0129           0.0064
stratified               mack   rolling 0.0130           0.0065
stratified                mcl      last 0.0297           0.0149
stratified                mcl    pooled 0.0117           0.0058
stratified                mcl   rolling 0.0118           0.0059
stratified tlrn_13_fh_factors      last 0.0490           0.0245
stratified tlrn_13_fh_factors    pooled 0.0180           0.0090
stratified tlrn_13_fh_factors   rolling 0.0178           0.0089

Sensitivity analysis, a stratified bootstrap (the dominant company in every resample once; it conditions on that company being present; not the verdict):
    method          benchmark statistic     diff      low    high  share_favour
tlrn_13_fh               mack      last -0.00519 -0.01486 0.00526        0.8429
tlrn_13_fh               mack    pooled  0.00486 -0.00131 0.01154        0.0653
tlrn_13_fh               mack   rolling  0.00515 -0.00112 0.01187        0.0569
tlrn_13_fh                mcl      last -0.00447 -0.02068 0.00902        0.7263
tlrn_13_fh                mcl    pooled  0.00465 -0.00084 0.01082        0.0507
tlrn_13_fh                mcl   rolling  0.00485 -0.00065 0.01111        0.0422
tlrn_13_fh tlrn_13_fh_factors      last -0.01714 -0.04234 0.00669        0.9126
tlrn_13_fh tlrn_13_fh_factors    pooled -0.00487 -0.01458 0.00347        0.8300
tlrn_13_fh tlrn_13_fh_factors   rolling -0.00446 -0.01357 0.00420        0.8041
   the same rule on the stratified intervals gives: NOT DISTINGUISHABLE

Comparison rows (row minus the main row; positive means the main row is better):
                   row    against statistic    diff      low    high  share_favour
           raw_network tlrn_13_fh      last 0.00625 -0.00223 0.04733        0.1147
           raw_network tlrn_13_fh    pooled 0.00946  0.00459 0.03498        0.0000
           raw_network tlrn_13_fh   rolling 0.00961  0.00509 0.03332        0.0000
    tlrn_13_fh_factors tlrn_13_fh      last 0.01714 -0.03481 0.08522        0.1696
    tlrn_13_fh_factors tlrn_13_fh    pooled 0.00487 -0.00547 0.06084        0.2594
    tlrn_13_fh_factors tlrn_13_fh   rolling 0.00446 -0.00623 0.05793        0.2852
 first_study:tlrn_8_ay tlrn_13_fh      last 0.00815 -0.04736 0.03637        0.3242
first_study:tlrn_13_ay tlrn_13_fh      last 0.00660 -0.05824 0.02514        0.4027

Median blend weight alpha over members and draws:
 valuation  alpha_median
         6        0.6089
         7        0.8135
         8        0.7128
        10        0.6079

Spread of the blend weight alpha over members and draws:
 valuation  alpha_min  alpha_q25  alpha_median  alpha_q75  alpha_max  alpha_std  alpha_count
         6     0.2715     0.4959        0.6089     0.7638     1.0000     0.1828          200
         7     0.2423     0.6630        0.8135     0.9334     1.0000     0.1774          200
         8     0.2866     0.5967        0.7128     0.8188     1.0000     0.1663          200
        10     0.3948     0.5393        0.6079     0.6814     0.8736     0.0983          200

VERDICT for tlrn_13_fh on cohort B (confirmatory of a fixed specification, not out-of-sample): NOT DISTINGUISHABLE
   vs mack: valuation 10 [-1.8735%, +3.5050%], pooled [-1.0720%, +2.1289%], valuation 10 without the dominant company +0.6109% [-2.6954%, +4.4065%]; vs mcl: valuation 10 [-4.8581%, +2.8596%], pooled [-1.6887%, +1.4583%], valuation 10 without the dominant company +0.1131% [-7.0214%, +3.5489%]; median-draw valuation-10 Pool_APE 5.0128% against mack 5.4810% and mcl 5.4098%; conditions i_last_below_both no, ii_pooled_below_both no, iii_median_draw_below_both yes, iv_excluding_dominant_not_worse no

Amendment 1, a separate question (not part of the verdict above): does the network add anything beyond the blend? Pool_APE difference, model minus the factors-only row, negative favours the model:
      model      compared_with statistic     diff      low     high  share_favour
 tlrn_13_fh tlrn_13_fh_factors      last -0.01714 -0.08522  0.03481        0.8304
 tlrn_13_fh tlrn_13_fh_factors   rolling -0.00446 -0.05793  0.00623        0.7148
 tlrn_13_fh tlrn_13_fh_factors    pooled -0.00487 -0.06084  0.00547        0.7406
 tlrn_13_fh tlrn_13_fh_factors excluding -0.01082 -0.10615  0.04335        0.6310
raw_network tlrn_13_fh_factors      last -0.01089 -0.06298  0.06959        0.6760
raw_network tlrn_13_fh_factors   rolling  0.00515 -0.03237  0.01315        0.3406
raw_network tlrn_13_fh_factors excluding  0.01860 -0.08830  0.07988        0.3575
 tlrn_13_fh        raw_network      last -0.00625 -0.04733  0.00223        0.8853
 tlrn_13_fh        raw_network   rolling -0.00961 -0.03332 -0.00509        1.0000
 tlrn_13_fh        raw_network    pooled -0.00946 -0.03498 -0.00459        1.0000
 tlrn_13_fh        raw_network excluding -0.02941 -0.05446 -0.01179        0.9984
NETWORK QUESTION for tlrn_13_fh on cohort B: NOT DISTINGUISHABLE
   valuation 10 [-8.5217%, +3.4806%]; rolling [-5.7932%, +0.6233%]; valuation 10 without the dominant company [-10.6151%, +4.3353%]; conditions a_last_below no, b_rolling_below no, c_excluding_not_above yes

Amendment 2, another question (outside the verdict): main row minus the raw network, by the same interval rules (negative favours the main row, i.e. the blend adds value):
BLEND QUESTION for tlrn_13_fh on cohort B: NOT DISTINGUISHABLE
   valuation 10 [-4.7329%, +0.2235%]; rolling [-3.3316%, -0.5088%]; valuation 10 without the dominant company [-5.4460%, -1.1792%]; conditions a_last_below no, b_rolling_below yes, c_excluding_not_above yes
(cohort B evaluated in 119s)
cohort C: 83 companies, 232 pairs, 83 scored; selection [675, 405, 322, 232]; hash 4a613cc4cb5f0f94; data 92f5d7a74f37a0c2; from cas-schedule-p publish 20260613_041006
  benchmarks fitted in 49s; refusals {'mack': 0, 'mcl': 0}
cohort C tlrn_13_fh: data hash differs (fits recorded e52d50e002d50e16, evaluator built 92f5d7a74f37a0c2); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
cohort C tlrn_13_fh_factors: data hash differs (fits recorded e52d50e002d50e16, evaluator built 92f5d7a74f37a0c2); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
  benchmarks fitted in 69s; refusals {'mack': 0, 'mcl': 0, 'sur': 0}
=== cohort C: 83 companies, 232 pairs, 83 scored; selection [675, 405, 322, 232]; hash 4a613cc4cb5f0f94; data 92f5d7a74f37a0c2; from cas-schedule-p publish 20260613_041006
status of any verdict on this cohort: out-of-sample in accident years only (not tuned on as far as we know, assumed not known; its companies and calendar years overlap A and B); valuation 10 is 1997-12-31
scored companies: 83 of the cohort's board 83; cells scored at valuation 10: 10440
dominant company 1767 holds 66.7% of the actual valuation-10 reserve
data hash, tlrn_13_fh: cohort C tlrn_13_fh: data hash differs (fits recorded e52d50e002d50e16, evaluator built 92f5d7a74f37a0c2); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
data hash, tlrn_13_fh_factors: cohort C tlrn_13_fh_factors: data hash differs (fits recorded e52d50e002d50e16, evaluator built 92f5d7a74f37a0c2); anchors, premiums, lines and companies of every fit equal the cohort's within tolerance, so the fits are scored
first study accident-year rows: not applicable: the first study ran cohorts A and B only
first study benchmarks at valuation 10: 
   mack max_relative_difference 8.24907e-13
   mcl max_relative_difference 8.24907e-13
   actual max_relative_difference 0
benchmark Pool_APE by valuation [6, 7, 8, 10]:
   mack: 13.772%  16.702%  16.923%  11.947%  | rolling 14.836%
   mcl: 13.836%  16.818%  17.044%  4.969%  | rolling 13.167%

--- raw_network: 10 draws
   Pool_APE per draw at valuation 6: min 21.043%  median 22.117%  max 24.198%  sd 1.185%
   Pool_APE per draw at valuation 7: min 19.927%  median 22.202%  max 23.106%  sd 1.144%
   Pool_APE per draw at valuation 8: min 10.218%  median 11.256%  max 12.834%  sd 0.799%
   Pool_APE per draw at valuation 10: min 13.841%  median 16.920%  max 19.315%  sd 1.557%
   Pool_APE per draw at valuation rolling: min 17.438%  median 18.355%  max 18.804%  sd 0.514%

--- tlrn_13_fh: 10 draws
   Pool_APE per draw at valuation 6: min 15.697%  median 16.092%  max 16.531%  sd 0.276%
   Pool_APE per draw at valuation 7: min 18.467%  median 19.450%  max 20.331%  sd 0.685%
   Pool_APE per draw at valuation 8: min 12.526%  median 13.230%  max 14.134%  sd 0.458%
   Pool_APE per draw at valuation 10: min 12.951%  median 15.886%  max 17.857%  sd 1.504%
   Pool_APE per draw at valuation rolling: min 15.329%  median 16.120%  max 16.847%  sd 0.508%

--- tlrn_13_fh_factors: 10 draws
   Pool_APE per draw at valuation 6: min 11.489%  median 11.586%  max 11.764%  sd 0.088%
   Pool_APE per draw at valuation 7: min 13.460%  median 13.713%  max 13.973%  sd 0.140%
   Pool_APE per draw at valuation 8: min 13.728%  median 13.942%  max 14.027%  sd 0.096%
   Pool_APE per draw at valuation 10: min 7.564%  median 7.754%  max 7.864%  sd 0.109%
   Pool_APE per draw at valuation rolling: min 11.667%  median 11.761%  max 11.798%  sd 0.049%

Pool_APE of the draw average, and the other reported metrics:
            method valuation    n  pool_ape  pool_pe  medape         mae  log_ratio  n_log    mape        wrmse  share_better_than_mcl  share_better_than_mack
              mack         6 83.0   0.13772  0.11747 0.12285 25651.72878    0.18873   82.0 0.25424 1.094100e+06                0.24096                     NaN
              mack         7 83.0   0.16702  0.15088 0.11861 31118.06083    0.17833   82.0 0.17934 1.432026e+06                0.28916                     NaN
              mack         8 83.0   0.16923  0.15798 0.13901 27411.99726    0.16263   82.0 0.16925 1.260691e+06                0.38554                     NaN
              mack        10 83.0   0.11947  0.11329 0.15402 28048.20227    0.19984   82.0 0.25035 1.251416e+06                0.59036                     NaN
              mack   rolling  NaN   0.14836  0.13490 0.13362 28057.49729    0.18238    NaN 0.21330 1.259558e+06                0.37651                     NaN
               mcl         6 83.0   0.13836  0.11693 0.12380 25770.91739    0.18505   82.0 0.22742 1.094100e+06                    NaN                 0.21687
               mcl         7 83.0   0.16818  0.15059 0.11885 31332.93733    0.18339   82.0 0.19199 1.432027e+06                    NaN                 0.22892
               mcl         8 83.0   0.17044  0.15433 0.13102 27608.01022    0.15637   81.0 0.18429 1.261443e+06                    NaN                 0.46988
               mcl        10 83.0   0.04969  0.03792 0.16878 11666.57142    0.22786   82.0 0.32398 3.333561e+05                    NaN                 0.39759
               mcl   rolling  NaN   0.13167  0.11494 0.13561 24094.60909    0.18817    NaN 0.23192 1.030232e+06                    NaN                 0.32831
       raw_network         6 83.0   0.22628  0.21115 0.22789 42146.38842    0.27704   82.0 0.33340 1.944898e+06                0.28916                 0.28916
       raw_network         7 83.0   0.21749  0.20533 0.21018 40521.17275    0.21988   82.0 0.24592 1.804762e+06                0.31325                 0.28916
       raw_network         8 83.0   0.11396  0.10345 0.12449 18458.35119    0.14464   82.0 0.15322 7.109527e+05                0.55422                 0.60241
       raw_network        10 83.0   0.17159  0.17105 0.29498 40285.37888    0.29605   82.0 0.37864 1.785266e+06                0.27711                 0.26506
       raw_network   rolling  NaN   0.18233  0.17274 0.21438 35352.82281    0.23440    NaN 0.27779 1.561470e+06                0.35843                 0.36145
        tlrn_13_fh         6 83.0   0.16108  0.14578 0.13691 30002.56528    0.18238   82.0 0.22325 1.349613e+06                0.50602                 0.45783
        tlrn_13_fh         7 83.0   0.19397  0.18119 0.14610 36139.54179    0.17761   82.0 0.19410 1.648021e+06                0.37349                 0.39759
        tlrn_13_fh         8 83.0   0.13300  0.12313 0.10888 21543.65767    0.14405   82.0 0.15083 9.157298e+05                0.61446                 0.61446
        tlrn_13_fh        10 83.0   0.16042  0.15977 0.28389 37661.42839    0.28568   82.0 0.36714 1.665213e+06                0.28916                 0.26506
        tlrn_13_fh   rolling  NaN   0.16212  0.15247 0.16894 31336.79828    0.19743    NaN 0.23383 1.394644e+06                0.44578                 0.43373
tlrn_13_fh_factors         6 83.0   0.11591  0.08417 0.12906 21589.06232    0.18439   82.0 0.22006 8.762504e+05                0.48193                 0.46988
tlrn_13_fh_factors         7 83.0   0.13723  0.10445 0.12075 25567.33147    0.18731   82.0 0.19420 1.115034e+06                0.46988                 0.48193
tlrn_13_fh_factors         8 83.0   0.13920  0.09845 0.14113 22547.16059    0.20469   82.0 0.20907 9.383619e+05                0.44578                 0.48193
tlrn_13_fh_factors        10 83.0   0.07741 -0.00686 0.26235 18174.35906    0.29077   82.0 0.41923 1.225636e+05                0.40964                 0.39759
tlrn_13_fh_factors   rolling  NaN   0.11744  0.07005 0.16332 21969.47836    0.21679    NaN 0.26064 7.630525e+05                0.45181                 0.45783

Difference in Pool_APE (row minus benchmark; negative favours the row), 95% interval:
            method benchmark    statistic     diff      low     high  share_favour
        tlrn_13_fh      mack         last  0.04095 -0.00002  0.10867        0.0251
        tlrn_13_fh      mack       pooled  0.01705  0.00054  0.05016        0.0229
        tlrn_13_fh      mack      rolling  0.01376 -0.00051  0.04333        0.0275
        tlrn_13_fh      mack  valuation 6  0.02336 -0.00993  0.04349           NaN
        tlrn_13_fh      mack  valuation 7  0.02695  0.02050  0.04721           NaN
        tlrn_13_fh      mack  valuation 8 -0.03623 -0.04273 -0.00446           NaN
        tlrn_13_fh      mack valuation 10  0.04095 -0.00002  0.10867           NaN
        tlrn_13_fh       mcl         last  0.11072  0.06366  0.12324        0.0000
        tlrn_13_fh       mcl       pooled  0.03766  0.01839  0.04807        0.0000
        tlrn_13_fh       mcl      rolling  0.03045  0.01305  0.04155        0.0000
        tlrn_13_fh       mcl  valuation 6  0.02272 -0.01087  0.03968           NaN
        tlrn_13_fh       mcl  valuation 7  0.02580  0.01610  0.04231           NaN
        tlrn_13_fh       mcl  valuation 8 -0.03744 -0.04275 -0.00479           NaN
        tlrn_13_fh       mcl valuation 10  0.11072  0.06366  0.12324           NaN
tlrn_13_fh_factors      mack         last -0.04206 -0.09343  0.19004        0.6249
tlrn_13_fh_factors      mack       pooled -0.03165 -0.05502  0.10242        0.7300
tlrn_13_fh_factors      mack      rolling -0.03092 -0.05498  0.09392        0.7513
tlrn_13_fh_factors      mack  valuation 6 -0.02181 -0.05944  0.04546           NaN
tlrn_13_fh_factors      mack  valuation 7 -0.02979 -0.07630  0.06256           NaN
tlrn_13_fh_factors      mack  valuation 8 -0.03003 -0.08417  0.09704           NaN
tlrn_13_fh_factors      mack valuation 10 -0.04206 -0.09343  0.19004           NaN
tlrn_13_fh_factors       mcl         last  0.02772 -0.01438  0.18886        0.1488
tlrn_13_fh_factors       mcl       pooled -0.01105 -0.03098  0.09893        0.6361
tlrn_13_fh_factors       mcl      rolling -0.01423 -0.03632  0.09052        0.6862
tlrn_13_fh_factors       mcl  valuation 6 -0.02245 -0.06059  0.04101           NaN
tlrn_13_fh_factors       mcl  valuation 7 -0.03095 -0.07989  0.05748           NaN
tlrn_13_fh_factors       mcl  valuation 8 -0.03124 -0.09091  0.09713           NaN
tlrn_13_fh_factors       mcl valuation 10  0.02772 -0.01438  0.18886           NaN
       raw_network      mack         last  0.05212  0.00916  0.12279        0.0115
       raw_network      mack       pooled  0.03793  0.00985  0.10125        0.0076
       raw_network      mack      rolling  0.03397  0.00817  0.09719        0.0086
       raw_network      mack  valuation 6  0.08856 -0.00052  0.18563           NaN
       raw_network      mack  valuation 7  0.05047  0.04120  0.10363           NaN
       raw_network      mack  valuation 8 -0.05528 -0.06667  0.00461           NaN
       raw_network      mack valuation 10  0.05212  0.00916  0.12279           NaN
       raw_network       mcl         last  0.12190  0.07380  0.13523        0.0000
       raw_network       mcl       pooled  0.05854  0.02791  0.09996        0.0000
       raw_network       mcl      rolling  0.05066  0.02139  0.09647        0.0004
       raw_network       mcl  valuation 6  0.08792 -0.00131  0.18135           NaN
       raw_network       mcl  valuation 7  0.04932  0.03897  0.09863           NaN
       raw_network       mcl  valuation 8 -0.05649 -0.06697  0.00526           NaN
       raw_network       mcl valuation 10  0.12190  0.07380  0.13523           NaN

Valuation-10 difference without the dominant company:
            method benchmark    diff      low    high  share_favour
        tlrn_13_fh      mack 0.04468 -0.00991 0.12264        0.0628
        tlrn_13_fh       mcl 0.08141  0.05742 0.11520        0.0000
tlrn_13_fh_factors      mack 0.09708  0.00442 0.21966        0.0182
tlrn_13_fh_factors       mcl 0.13381  0.04986 0.20889        0.0023
       raw_network      mack 0.05562 -0.00135 0.13744        0.0287
       raw_network       mcl 0.09235  0.06815 0.13118        0.0000

Pool_APE within quartiles of the actual reserve:
method                    mack      mcl  raw_network  tlrn_13_fh  tlrn_13_fh_factors
valuation quartile n                                                                
6         1        21  0.23117  0.23874      0.39309     0.22729             0.23440
          2        21  0.16273  0.16577      0.27972     0.15236             0.14561
          3        21  0.19169  0.18517      0.28322     0.18843             0.15962
          4        20  0.13528  0.13612      0.22318     0.16004             0.11369
7         1        21  0.18001  0.17626      0.23913     0.15552             0.17748
          2        21  0.16372  0.15675      0.23646     0.17466             0.17599
          3        21  0.14957  0.18340      0.23166     0.19676             0.18563
          4        20  0.16761  0.16777      0.21671     0.19424             0.13497
8         1        21  0.19288  0.22205      0.15768     0.15804             0.27213
          2        21  0.14041  0.13298      0.14453     0.13326             0.16248
          3        21  0.14658  0.11802      0.13564     0.12240             0.16536
          4        20  0.17024  0.17242      0.11275     0.13326             0.13766
10        1        21  0.30565  0.37574      0.54512     0.52127             0.44746
          2        21  0.17657  0.19973      0.30594     0.29129             0.24763
          3        21  0.16185  0.20366      0.34387     0.32107             0.29577
          4        20  0.11690  0.04202      0.16337     0.15265             0.06728

Precision (Amendment 2): the width of each 95% interval and the gain it could just detect, about half the width, in Pool_APE (0.03 is three points). A gain of half a point (0.005) is far below these, so 'not distinguishable' is the expected result on one cohort:
 bootstrap          benchmark statistic  width  detectable_gain
   uniform               mack      last 0.1087           0.0543
   uniform               mack    pooled 0.0496           0.0248
   uniform               mack   rolling 0.0438           0.0219
   uniform                mcl      last 0.0596           0.0298
   uniform                mcl    pooled 0.0297           0.0148
   uniform                mcl   rolling 0.0285           0.0143
stratified               mack      last 0.0431           0.0216
stratified               mack    pooled 0.0189           0.0094
stratified               mack   rolling 0.0169           0.0085
stratified                mcl      last 0.0231           0.0116
stratified                mcl    pooled 0.0132           0.0066
stratified                mcl   rolling 0.0127           0.0063
stratified tlrn_13_fh_factors      last 0.0812           0.0406
stratified tlrn_13_fh_factors    pooled 0.0362           0.0181
stratified tlrn_13_fh_factors   rolling 0.0370           0.0185

Sensitivity analysis, a stratified bootstrap (the dominant company in every resample once; it conditions on that company being present; not the verdict):
    method          benchmark statistic    diff     low    high  share_favour
tlrn_13_fh               mack      last 0.04095 0.01828 0.06142        0.0000
tlrn_13_fh               mack    pooled 0.01705 0.00738 0.02625        0.0000
tlrn_13_fh               mack   rolling 0.01376 0.00518 0.02212        0.0001
tlrn_13_fh                mcl      last 0.11072 0.09976 0.12288        0.0000
tlrn_13_fh                mcl    pooled 0.03766 0.03125 0.04450        0.0000
tlrn_13_fh                mcl   rolling 0.03045 0.02446 0.03711        0.0000
tlrn_13_fh tlrn_13_fh_factors      last 0.08300 0.04483 0.12607        0.0000
tlrn_13_fh tlrn_13_fh_factors    pooled 0.04870 0.02878 0.06499        0.0000
tlrn_13_fh tlrn_13_fh_factors   rolling 0.04468 0.02471 0.06168        0.0000
   the same rule on the stratified intervals gives: WORSE

Comparison rows (row minus the main row; positive means the main row is better):
               row    against statistic     diff      low    high  share_favour
       raw_network tlrn_13_fh      last  0.01118  0.00828 0.01568        0.0000
       raw_network tlrn_13_fh    pooled  0.02088  0.00807 0.05456        0.0022
       raw_network tlrn_13_fh   rolling  0.02021  0.00722 0.05727        0.0037
tlrn_13_fh_factors tlrn_13_fh      last -0.08300 -0.13477 0.10092        0.6610
tlrn_13_fh_factors tlrn_13_fh    pooled -0.04870 -0.06682 0.05619        0.8002
tlrn_13_fh_factors tlrn_13_fh   rolling -0.04468 -0.06210 0.05486        0.8136

Median blend weight alpha over members and draws:
 valuation  alpha_median
         6        0.3466
         7        0.4805
         8        0.6782
        10        0.9361

Spread of the blend weight alpha over members and draws:
 valuation  alpha_min  alpha_q25  alpha_median  alpha_q75  alpha_max  alpha_std  alpha_count
         6     0.0000     0.2219        0.3466     0.4487     0.7835     0.1665          200
         7     0.2052     0.3908        0.4805     0.5796     1.0000     0.1427          200
         8     0.3465     0.5697        0.6782     0.7905     1.0000     0.1494          200
        10     0.5323     0.8449        0.9361     1.0000     1.0000     0.1062          200

VERDICT for tlrn_13_fh on cohort C (out-of-sample in accident years only (not tuned on as far as we know, assumed not known; its companies and calendar years overlap A and B)): WORSE
   vs mack: valuation 10 [-0.0015%, +10.8673%], pooled [+0.0538%, +5.0159%], valuation 10 without the dominant company +4.4681% [-0.9914%, +12.2643%]; vs mcl: valuation 10 [+6.3661%, +12.3242%], pooled [+1.8392%, +4.8068%], valuation 10 without the dominant company +8.1414% [+5.7419%, +11.5200%]; median-draw valuation-10 Pool_APE 15.8857% against mack 11.9470% and mcl 4.9693%; conditions i_last_below_both no, ii_pooled_below_both no, iii_median_draw_below_both no, iv_excluding_dominant_not_worse no

Amendment 1, a separate question (not part of the verdict above): does the network add anything beyond the blend? Pool_APE difference, model minus the factors-only row, negative favours the model:
      model      compared_with statistic     diff      low     high  share_favour
 tlrn_13_fh tlrn_13_fh_factors      last  0.08300 -0.10092  0.13477        0.3390
 tlrn_13_fh tlrn_13_fh_factors   rolling  0.04468 -0.05486  0.06210        0.1864
 tlrn_13_fh tlrn_13_fh_factors    pooled  0.04870 -0.05619  0.06682        0.1998
 tlrn_13_fh tlrn_13_fh_factors excluding -0.05240 -0.11660  0.03385        0.9044
raw_network tlrn_13_fh_factors      last  0.09418 -0.08913  0.14647        0.3178
raw_network tlrn_13_fh_factors   rolling  0.06489 -0.01291  0.07945        0.0531
raw_network tlrn_13_fh_factors excluding -0.04146 -0.10506  0.04732        0.8521
 tlrn_13_fh        raw_network      last -0.01118 -0.01568 -0.00828        1.0000
 tlrn_13_fh        raw_network   rolling -0.02021 -0.05727 -0.00722        0.9963
 tlrn_13_fh        raw_network    pooled -0.02088 -0.05456 -0.00807        0.9978
 tlrn_13_fh        raw_network excluding -0.01094 -0.01768 -0.00772        1.0000
NETWORK QUESTION for tlrn_13_fh on cohort C: NOT DISTINGUISHABLE
   valuation 10 [-10.0917%, +13.4768%]; rolling [-5.4856%, +6.2104%]; valuation 10 without the dominant company [-11.6605%, +3.3854%]; conditions a_last_below no, b_rolling_below no, c_excluding_not_above yes

Amendment 2, another question (outside the verdict): main row minus the raw network, by the same interval rules (negative favours the main row, i.e. the blend adds value):
BLEND QUESTION for tlrn_13_fh on cohort C: BLEND ADDS VALUE
   valuation 10 [-1.5685%, -0.8276%]; rolling [-5.7272%, -0.7218%]; valuation 10 without the dominant company [-1.7679%, -0.7719%]; conditions a_last_below yes, b_rolling_below yes, c_excluding_not_above yes
(cohort C evaluated in 121s)

VERDICTS
  A: not distinguishable [confirmatory of a fixed specification, not out-of-sample] (10 draws)
     network question: not distinguishable
     blend question: not distinguishable
     stratified sensitivity: not distinguishable
  B: not distinguishable [confirmatory of a fixed specification, not out-of-sample] (10 draws)
     network question: not distinguishable
     blend question: not distinguishable
     stratified sensitivity: not distinguishable
  C: worse [out-of-sample in accident years only (not tuned on as far as we know, assumed not known; its companies and calendar years overlap A and B)] (10 draws)
     network question: not distinguishable
     blend question: blend adds value
     stratified sensitivity: worse
OVERALL (Amendment 2, B5): worse on the fresh cohort (C worse; A not distinguishable, B not distinguishable)
