tools/gemv/tile_layout_race.py, box 10.2.0.206 GPU0 (RTX 4090, sm_89) locked 1800 MHz,
registry.arbi.work/arbi-serve:test-latest (2026-09-18), torch 2.12.1+cu130,
base integ/gguf-all 662b94a4f.  CUDA_VISIBLE_DEVICES=0 throughout; card 1 held
another tenant's 23.9 GB and was never addressed.

Columns: median burst microseconds over rot=4 distinct weights x inner=6 calls,
behind a read-only 256 MiB L2 scrub, arms shuffled inside every rep.
nib/bit and ctl are TIMES (below 1.000 is faster).  live/null is the whole
decode's share of the step.

== witness (all 8 routable served cells) ==
  down_proj    17408x5120   bitstream: EQUAL to the oracle at every probed k
  down_proj    17408x5120   nibble   : EQUAL to the oracle at every probed k
  down_proj    17408x5120   M=1 dense:  EQUAL max|d|=0 finite 5120/5120
  down_proj    17408x5120   M=2 dense:  EQUAL max|d|=0 finite 10240/10240
  gate/up       5120x17408  bitstream: EQUAL to the oracle at every probed k
  gate/up       5120x17408  nibble   : EQUAL to the oracle at every probed k
  gate/up       5120x17408  M=1 dense:  EQUAL max|d|=0 finite 17408/17408
  gate/up       5120x17408  M=2 dense:  EQUAL max|d|=0 finite 34816/34816
  q_proj        5120x12288  bitstream: EQUAL to the oracle at every probed k
  q_proj        5120x12288  nibble   : EQUAL to the oracle at every probed k
  q_proj        5120x12288  M=1 dense:  EQUAL max|d|=0 finite 12288/12288
  q_proj        5120x12288  M=2 dense:  EQUAL max|d|=0 finite 24576/24576
  mtp_fc       10240x5120   bitstream: EQUAL to the oracle at every probed k
  mtp_fc       10240x5120   nibble   : EQUAL to the oracle at every probed k
  mtp_fc       10240x5120   M=1 dense:  EQUAL max|d|=0 finite 5120/5120
  mtp_fc       10240x5120   M=2 dense:  EQUAL max|d|=0 finite 10240/10240
  qkv           5120x10240  bitstream: EQUAL to the oracle at every probed k
  qkv           5120x10240  nibble   : EQUAL to the oracle at every probed k
  qkv           5120x10240  M=1 dense:  EQUAL max|d|=0 finite 10240/10240
  qkv           5120x10240  M=2 dense:  EQUAL max|d|=0 finite 20480/20480
  attn_output   6144x5120   bitstream: EQUAL to the oracle at every probed k
  attn_output   6144x5120   nibble   : EQUAL to the oracle at every probed k
  attn_output   6144x5120   M=1 dense:  EQUAL max|d|=0 finite 5120/5120
  attn_output   6144x5120   M=2 dense:  EQUAL max|d|=0 finite 10240/10240
  in_proj_z     5120x6144   bitstream: EQUAL to the oracle at every probed k
  in_proj_z     5120x6144   nibble   : EQUAL to the oracle at every probed k
  in_proj_z     5120x6144   M=1 dense:  EQUAL max|d|=0 finite 6144/6144
  in_proj_z     5120x6144   M=2 dense:  EQUAL max|d|=0 finite 12288/12288
  kv_proj       5120x1024   bitstream: EQUAL to the oracle at every probed k
  kv_proj       5120x1024   nibble   : EQUAL to the oracle at every probed k
  kv_proj       5120x1024   M=1 dense:  EQUAL max|d|=0 finite 1024/1024
  kv_proj       5120x1024   M=2 dense:  EQUAL max|d|=0 finite 2048/2048
  exit 0

== race, rot=4 reps=9 inner=6 splits=shipped seed=0 ==
cell              K      N  M  bitstream     nibble       null bitstream2  nib/bit     ctl live/null
down_proj     17408   5120  1    1594.37    1526.78    1515.52    1595.30   0.9576  1.0006    1.0520
down_proj     17408   5120  2    1874.98    1732.74    1618.94    1876.70   0.9241  1.0009    1.1581
gate/up        5120  17408  1    1552.19    1509.38    1512.45    1553.41   0.9724  1.0008    1.0263
gate/up        5120  17408  2    1801.22    1690.62    1603.58    1802.24   0.9386  1.0006    1.1232
q_proj         5120  12288  1    1186.82    1146.88    1125.38    1186.82   0.9664  1.0000    1.0546
q_proj         5120  12288  2    1376.26    1291.26    1220.29    1377.28   0.9382  1.0007    1.1278
mtp_fc        10240   5120  1     998.46     960.51     945.15     998.40   0.9620  0.9999    1.0564
mtp_fc        10240   5120  2    1168.58    1087.49    1025.02    1168.42   0.9306  0.9999    1.1400
qkv            5120  10240  1     998.11     957.09     941.06     996.35   0.9589  0.9982    1.0606
qkv            5120  10240  2    1169.41    1085.44    1017.86    1170.56   0.9282  1.0010    1.1489
attn_output    6144   5120  1     624.42     603.01     543.74     622.59   0.9657  0.9971    1.1484
attn_output    6144   5120  2     732.16     703.52     653.06     730.98   0.9609  0.9984    1.1211
in_proj_z      5120   6144  1     675.74     633.86     587.78     675.84   0.9380  1.0001    1.1497
in_proj_z      5120   6144  2     799.74     746.27     708.61     799.71   0.9331  1.0000    1.1286
kv_proj        5120   1024  1     221.18     215.04     207.87     222.14   0.9722  1.0043    1.0640
kv_proj        5120   1024  2     307.20     300.93     279.55     307.01   0.9796  0.9994    1.0989

== race, rot=4 reps=11 inner=6 splits=shipped seed=101 (a second process) ==
cell              K      N  M  bitstream     nibble       null bitstream2  nib/bit     ctl live/null
down_proj     17408   5120  1    1595.26    1533.95    1520.58    1594.37   0.9616  0.9994    1.0491
down_proj     17408   5120  2    1875.97    1732.64    1620.99    1875.97   0.9236  1.0000    1.1573
gate/up        5120  17408  1    1553.41    1509.50    1514.50    1552.38   0.9717  0.9993    1.0257
gate/up        5120  17408  2    1801.22    1691.65    1604.86    1803.26   0.9392  1.0011    1.1223
q_proj         5120  12288  1    1185.66    1145.86    1124.35    1184.77   0.9664  0.9992    1.0545
q_proj         5120  12288  2    1376.26    1291.26    1220.61    1374.21   0.9382  0.9985    1.1275
mtp_fc        10240   5120  1    1000.35     961.25     947.20     999.42   0.9609  0.9991    1.0561
mtp_fc        10240   5120  2    1169.44    1089.22    1025.02    1170.43   0.9314  1.0008    1.1409
qkv            5120  10240  1     995.33     955.39     941.06     995.33   0.9599  1.0000    1.0577
qkv            5120  10240  2    1170.43    1084.42    1019.71    1169.41   0.9265  0.9991    1.1478
attn_output    6144   5120  1     623.42     603.01     545.44     623.62   0.9673  1.0003    1.1430
attn_output    6144   5120  2     731.14     702.46     650.24     730.11   0.9608  0.9986    1.1244
in_proj_z      5120   6144  1     674.82     633.86     587.65     674.78   0.9393  1.0000    1.1483
in_proj_z      5120   6144  2     799.58     744.16     708.61     798.72   0.9307  0.9989    1.1284
kv_proj        5120   1024  1     221.18     215.94     206.85     221.18   0.9763  1.0000    1.0693
kv_proj        5120   1024  2     306.24     300.70     279.55     306.18   0.9819  0.9998    1.0955

== the grid control: the same race with the k split count FORCED equal ==
rot=4 reps=7 inner=6 splits=32 seed=0
down_proj     17408   5120  1    1583.10    1525.76    1553.41    1584.13   0.9638  1.0006    1.0191
down_proj     17408   5120  2    1835.01    1734.62    1639.10    1833.98   0.9453  0.9994    1.1195
gate/up        5120  17408  1    1592.32    1515.52    1479.68    1590.27   0.9518  0.9987    1.0761
gate/up        5120  17408  2    1869.73    1769.47    1661.82    1871.65   0.9464  1.0010    1.1251
in_proj_z      5120   6144  1     638.94     612.35     559.10     636.93   0.9584  0.9968    1.1428
in_proj_z      5120   6144  2     757.76     715.78     684.03     758.46   0.9446  1.0009    1.1078
rot=4 reps=7 inner=6 splits=64 seed=0
down_proj     17408   5120  1    1590.05    1535.26    1533.82    1589.25   0.9655  0.9995    1.0367
down_proj     17408   5120  2    1868.96    1767.42    1672.19    1867.46   0.9457  0.9992    1.1177
gate/up        5120  17408  1    1656.83    1574.91    1487.87    1654.91   0.9506  0.9988    1.1136
gate/up        5120  17408  2    2022.40    1914.59    1792.00    2022.18   0.9467  0.9999    1.1286
in_proj_z      5120   6144  1     676.54     644.10     591.87     675.84   0.9520  0.9990    1.1431
in_proj_z      5120   6144  2     831.49     790.43     752.64     831.49   0.9506  1.0000    1.1048

== ptxas, tools/gemv/compile_gate.py --arch sm_89 ==
sm_89: 87 kernel variants, no spill and no local memory
 regs  spill   lmem  symbol
   43      0      0  ag_gemv_kernel<GgufGemvLoader<4,32,256,0,true,false,false,8>, 4, false, 1>
   42      0      0  ag_gemv_kernel<AgNibbleTileTwin<GgufGemvLoader<4,32,256,0,true,false,false,8>>, 4, false, 1>
   38      0      0  ag_gemv_kernel<GgufGemvLoader<4,32,256,0,true,false,true,8>, 4, false, 1>   (ILV)
   38      0      0  ag_gemv_kernel<AgNibbleTileTwin<GgufGemvLoader<4,32,256,0,true,false,true,8>>, 4, false, 1>

== the pack, tools/gguf/tile_layout_reach.py ==
manifest /mnt/k8scache/tmp/gguf-out-27b/manifest.json
source blob sha256 3f227079003add2511437e5b1e94812e363385225bf6a9b47b0054a72bc8b01e
(the same digest today's live repack cache locks under, /mnt/k8scache/cache/arbi-serve/gguf-repack/.g5-3f227079...lock)

type      tensors          weights  w share   B/wt       bytes read  B share       tile
Q5_K          191   11,529,093,120  46.53%  0.703    8,106,393,600  50.40%  bitstream
IQ4_XS         70    5,882,511,360  23.74%  0.539    3,171,041,280  19.71%     nibble
Q4_K           68    4,168,089,600  16.82%  0.578    2,409,676,800  14.98%    MOVABLE
Q6_K           55    2,217,738,240   8.95%  0.820    1,819,238,400  11.31%  bitstream
IQ4_NL          6      498,073,600   2.01%  0.562      280,166,400   1.74%     nibble
Q3_K            3      267,386,880   1.08%  0.445      119,070,720   0.74%  bitstream
Q8_0          110      123,207,680   0.50%  1.062      130,908,160   0.81%  bitstream
IQ3_S           1       89,128,960   0.36%  0.539       48,046,080   0.30%     nibble
TOTAL         504   24,775,229,440 100.00%          16,084,541,440 100.00%

widening the nibble tile moves ['Q4_K']
  prefill GEMM reach, by weight: 16.8236%
  decode GEMV reach,  by bytes : 14.9813%
already a nibble tile ['IQ3_S', 'IQ4_NL', 'IQ4_XS']: 21.76% of decode bytes

Q4_K tensors per cell (the weights used to compose the per-cell margins):
  gate/up 19 of 130   down_proj 5 of 65   qkv 23 of 48   in_proj_z 13 of 48
  attn_output 1 of 65   q_proj 6 of 17   kv_proj 1 of 34   mtp_fc 0 of 1
  in_proj_ab 0 of 96 (N=48, below GEMV_TILE_ALIGN, never routed here)

== the packers, one thread, same fields, alternating, best of 9 ==
fields packed per call: 4,194,304
  bitstream    31.85 ms      131.7 Mweight/s
  nibble       12.21 ms      343.4 Mweight/s
  nibble/bitstream: 0.384x

== the box ==
before  07:23:12Z  up 85 days  load 0.99 3.05 2.65  75 GB avail
        GPU0  24 MiB / 0% / 210 MHz / 32 W / 36 C   (then locked 1800 MHz)
        GPU1  23898 MiB, another tenant, never addressed
        40 containers
only containers named gemvnib-* were created, and only those were removed.
