[MOUNT@a7b0bcb1-arbi/bf16] === arbi bf16 | /models/Qwen3.5-0.8B | ISL=16384 OSL=1024 | lock=2500MHz bits=n/a(TKV_BITS) | TP=1 gmu=0.99 mbt=8192 | mode=mount code=a7b0bcb1 img=registry.arbi.work/arbi-serve:latest@sha256:ff14ef5890ec46424d149d6088b2e1b08294b30571f0a24ad2b847a9ae0aadff | max_running_req=? ===
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Request throughput (req/s):              1.69      
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Output token throughput (tok/s):         1734.81   
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Peak output token throughput (tok/s):    2985.00   
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Median TTFT (ms):                        972.57    
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] P95 TTFT (ms):                           14324.43  
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Mean TPOT (ms):                          33.37     
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] Median TPOT (ms):                        35.57     
[MOUNT@a7b0bcb1-arbi/bf16] [c=64] P95 TPOT (ms):                           36.10     
[rig] GPU 0: lock HELD @ 2500MHz across 23 working samples (peak host load1 1.68, peak mem 23132 MiB, no throttle).
[rig] host=dev-gpu load1=1.39/32c | gpu0: sm=2505MHz util=100% mem=1MiB throttle=0x0000000000000000 pcie=gen4x16
[MOUNT@a7b0bcb1-arbi/tkv] === arbi tkv | /models/Qwen3.5-0.8B | ISL=16384 OSL=1024 | lock=2500MHz bits=4(TKV_BITS) | TP=1 gmu=0.99 mbt=8192 | mode=mount code=a7b0bcb1 img=registry.arbi.work/arbi-serve:latest@sha256:ff14ef5890ec46424d149d6088b2e1b08294b30571f0a24ad2b847a9ae0aadff | max_running_req=? ===
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Request throughput (req/s):              2.10      
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Output token throughput (tok/s):         2150.71   
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Peak output token throughput (tok/s):    4821.00   
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Median TTFT (ms):                        1178.48   
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] P95 TTFT (ms):                           14899.17  
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Mean TPOT (ms):                          26.04     
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] Median TPOT (ms):                        28.40     
[MOUNT@a7b0bcb1-arbi/tkv] [c=64] P95 TPOT (ms):                           28.89     
[rig] GPU 0: lock HELD @ 2500MHz across 20 working samples (peak host load1 16.28, peak mem 22832 MiB, no throttle).
[rig] host=dev-gpu load1=14.17/32c | gpu0: sm=2505MHz util=100% mem=1MiB throttle=0x0000000000000000 pcie=gen4x16
[MOUNT@49bae270-vllm/bf16] === vllm bf16 | /models/Qwen3.5-0.8B | ISL=16384 OSL=1024 | lock=2500MHz bits=n/a(TKV_BITS) | TP=1 gmu=0.95 mbt=8192 | mode=mount code=49bae270 img=registry.arbi.work/vllm-tkv:latest@sha256:895c53bbedc4f4ab9dafa3e1608adfe73b16b2602f842210defc3fe288d57121 | max_running_req=? ===
[MOUNT@49bae270-vllm/bf16] [c=64] Request throughput (req/s):              1.67      
[MOUNT@49bae270-vllm/bf16] [c=64] Output token throughput (tok/s):         1707.91   
[MOUNT@49bae270-vllm/bf16] [c=64] Peak output token throughput (tok/s):    3300.00   
[MOUNT@49bae270-vllm/bf16] [c=64] Median TTFT (ms):                        2385.58   
[MOUNT@49bae270-vllm/bf16] [c=64] P95 TTFT (ms):                           19918.45  
[MOUNT@49bae270-vllm/bf16] [c=64] Mean TPOT (ms):                          30.29     
[MOUNT@49bae270-vllm/bf16] [c=64] Median TPOT (ms):                        33.14     
[MOUNT@49bae270-vllm/bf16] [c=64] P95 TPOT (ms):                           33.42     
[rig] GPU 0: lock HELD @ 2500MHz across 28 working samples (peak host load1 14.52, peak mem 20692 MiB, no throttle).
[rig] host=dev-gpu load1=5.33/32c | gpu0: sm=2505MHz util=94% mem=1MiB throttle=0x0000000000000000 pcie=gen4x16
[MOUNT@49bae270-vllm/tkv] === vllm tkv | /models/Qwen3.5-0.8B | ISL=16384 OSL=1024 | lock=2500MHz bits=4(TKV_BITS) | TP=1 gmu=0.95 mbt=8192 | mode=mount code=49bae270 img=registry.arbi.work/vllm-tkv:latest@sha256:895c53bbedc4f4ab9dafa3e1608adfe73b16b2602f842210defc3fe288d57121 | max_running_req=? ===
[MOUNT@49bae270-vllm/tkv] [c=64] Request throughput (req/s):              2.30      
[MOUNT@49bae270-vllm/tkv] [c=64] Output token throughput (tok/s):         2350.68   
[MOUNT@49bae270-vllm/tkv] [c=64] Peak output token throughput (tok/s):    5184.00   
[MOUNT@49bae270-vllm/tkv] [c=64] Median TTFT (ms):                        1147.22   
[MOUNT@49bae270-vllm/tkv] [c=64] P95 TTFT (ms):                           13557.68  
[MOUNT@49bae270-vllm/tkv] [c=64] Mean TPOT (ms):                          23.78     
[MOUNT@49bae270-vllm/tkv] [c=64] Median TPOT (ms):                        25.95     
[MOUNT@49bae270-vllm/tkv] [c=64] P95 TPOT (ms):                           26.05     
[rig] GPU 0: lock HELD @ 2500MHz across 20 working samples (peak host load1 7.18, peak mem 20764 MiB, no throttle).
[rig] host=dev-gpu load1=1.03/32c | gpu0: sm=2505MHz util=0% mem=1MiB throttle=0x0000000000000000 pcie=gen4x16
