Pick a question, a finding or a run to inspect it
main · 8 recorded runs · changed 2 h ago · drawing {{N}}
Open
Does INT8 post-training quantization keep ResNet-50 within 1 point of FP32 top-1?
The record does not answer it yet: the FP32 baseline is not reproduced, and 2 of 6 criteria are not met.
2 of 6 criteria were declared after their evidence existed.
6 of 8 runs do not record the command they were run with.
Read them as a description of what happened, not as a test.
Does INT8 post-training quantization keep ResNet-50 within 1 point of FP32 top-1? open · 4 met · 1 not met
↳ How many calibration images does INT8 need? open · 1 not met
rb looks at out/, results/ and the git history; last looked 2026-10-19 14:20 UTC.
Research
2 results to review
fp32 top1 matches the published 76.13 c23 runs · 3 seeds per arm · 1 run not counted
rule: change.top1 ≥ -0.01 (n ≥ 3)
Recorded interpretation
the ONNX export keeps the INT8 reading · not confirmed
criterion met · loss
read in imagenet-val
declared 2026-10-12 · experiment e1 · metric top1
not reproduced · measured
criterion met · measured
read through a1
criterion not met · point only
rests on runsql7
Experiments
design confirmed, then changed: rb confirm e1 lists it
ready: code, environment and inputs recorded
git checkout 3f2a9c1 python eval.py --precision int8 --seed 1
| run | arms | top1 |
|---|---|---|
| runs1 | fp32 and int8 · seed 1 | 0.7598 · 0.7569 |
| runs3 | fp32 and int8 · seed 3 | 0.7599 · 0.7572 |
design not confirmed: rb confirm e2 lists it
Record
fp32: top1=0.7598 onnx_top1=0.7569 · int8: top1=0.7569 onnx_top1=0.7566
counts toward c1, c2, c3 · made ckpt-int8-s1 · read imagenet-val
python eval.py --precision int8 --seed 1
replaced by runs2b
fp32: latency_ms=6.1 · int8: latency_ms=3.9
4 calls wait on you.
Not reproduced · estimate 0.7597 outside 0.7613 ± 0.001
It rests on 3 runs · 1 run not counted · the whole finding
Its review will read:
rb decide c2 accept --why '‹why›'
rb decide c2 reject --why '‹why›'
rb decide c2 investigate --why '‹why›'
calib_images 64: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 128: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 192: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 256: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 320: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 384: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 448: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 512: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 576: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 640: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 704: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 768: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 832: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 896: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 960: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 1024: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 1088: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 1152: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 1216: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 1280: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 1344: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 1408: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 1472: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 1536: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 1600: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 1664: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 1728: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 1792: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 1856: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 1920: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 1984: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 2048: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 2112: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 2176: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 2240: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 2304: int8 top1 within 1 point of fp32 in 3 of 3 seeds
calib_images 2368: int8 top1 within 1 point of fp32 in 0 of 3 seeds
calib_images 2432: int8 top1 within 1 point of fp32 in 1 of 3 seeds
calib_images 2496: int8 top1 within 1 point of fp32 in 2 of 3 seeds
calib_images 2560: int8 top1 within 1 point of fp32 in 3 of 3 seeds
Its review will read:
rb decide e2 continue --why '‹why›'
rb decide e2 park --why '‹why›'
rb decide e2 stop --why '‹why›'
accept c3: the ONNX export holds onnx_top1
note: one run is enough for an export check · not confirmed
rb decide c3 accept --why 'one run is enough for an export check'
Confirm it in your own terminal:
rb confirm d7k2pqa
park e2, as notes.md:3 reads: "we park the 128-image cell for now"
Confirm it in your own terminal:
rb confirm d4m8xqe