rb.research record · int8-resnet50/INT8 ResNet-50 on ImageNetRerun this research2026-10-19 14:20 UTC · rb 0.9.0About

INT8 ResNet-50 on ImageNet

2 questions · 2 experiments · 6 criteria · 9 runs, 1 retracted · 4 resources

Where the work stands

2 of 6 criteria were declared after their evidence existed · 2 of 8 runs record their command (as given)

All criteria · 6: 4 criteria met · 2 criteria not met

Your call (4)

4 calls wait on you.

Review the calls waiting on you →

Waiting for your confirmation, listed apart

Proposed, not made (1)

  • accept c3: the ONNX export holds onnx_top1

    note: one run is enough for an export check · not confirmed

    rb decide c3 accept --why 'one run is enough for an export check'

    Confirm it in your own terminal:

    rb confirm d7k2pqa

Past calls mentioned, not confirmed (1)

  • park e2, as notes.md:3 reads: "we park the 128-image cell for now"

    Confirm it in your own terminal:

    rb confirm d4m8xqe

Reproduce this project

Everything on this page was computed from the record this bundle carries. Rerun that computation, and the runs behind it, on your machine:

Rerun this research

Recent calls and changes

Every change, newest first →

Notes on the project (1)
  • 2026-10-12 UTC: the validation images are not in the calibration set

Research

Research

Does INT8 post-training quantization keep ResNet-50 within 1 point of FP32 top-1?

2 results to review

fp32 top1 matches the published 76.13 c2

Recorded interpretation

INT8 keeps ResNet-50 within the one-point budget

1 more interpretation

the ONNX export keeps the INT8 reading · not confirmed

1 run not counted
qualification

1 run not counted

runs2 was retracted and replaced by runs2b.

All criteria

c1 int8 holds top1 within 1 point of fp32 · criterion met
criterionc1in e1

int8 holds top1 within 1 point of fp32

criterion met · loss

-0.0029 [-0.0071, +0.0013] top1 · paired difference, t interval, 3 seeds, 95% (rb)

what it rests on

  • runs1 fp32 and int8 · seed 1 · counted
  • runs2b fp32 and int8 · seed 2 · counted
  • runs2 fp32 and int8 · seed 2 · not counted: retracted
  • runs3 fp32 and int8 · seed 3 · counted

read in imagenet-val

receipt

declared 2026-10-12 · experiment e1 · metric top1 · what the words mean

c2 fp32 top1 matches the published 76.13 · not reproduced
criterionc2in e1

fp32 top1 matches the published 76.13

not reproduced · measured

0.7597

what it rests on

Your call on c2

c3 the ONNX export holds onnx_top1 · criterion met
criterionc3

the ONNX export holds onnx_top1

criterion met · point only

rests on runs1

c4 the per-class top1 drop stays under 2 points · criterion met
criterionc4

the per-class top1 drop stays under 2 points for at least 99% of classes

criterion met · measured

read through a1

c5 int8 stays within 1 point of fp32 at every calibration size · criterion not met
criterionc5

int8 stays within 1 point of fp32 at every calibration size

gate missed · point only

rests on runc128, runc512

c6 int8 latency is at most 0.6 of fp32 on the T4 · criterion not met
criterionc6

int8 latency is at most 0.6 of fp32 on the T4

criterion not met · point only

rests on runsql7

Experiments

Experiments

int8 against fp32

design confirmed, then changed: rb confirm e1 lists it

Reproduce

ready: code, environment and inputs recorded

Rerun this research with Rabbit Brain (rb). Its record: https://example.org/int8/page.html

1. Install rb: pip install rabbit-brain
2. Clone the record into a new folder: rb clone https://example.org/int8/bundle.zip int8-resnet50
   Where the record names the code the runs need, rb clone says its repository and commit. If it names none, ask me where the code is.
3. In that folder, run rb context and read it before anything else. It names each resource, such as a model or a dataset, with how to get it.
4. Fetch that repository and check out its commit in the same folder, beside .rb/, leaving out its own .rb/ and the results its runs wrote (its results folders and databases). Get those resources. Rerun experiment e1, "int8 against fp32", the way the record describes it, with rb journal run -- <command>, and record each run on its experiment: rb run add <experiment> --from <the file it wrote> --command "<that command>" (rb says any per-run value it needs).
5. Then run rb status, and tell me which of its rules (c1, c2) reproduced and which did not, with their numbers. Do not change a rule. If something cannot run here, say what and why.
The whole recipe as one script
git checkout 3f2a9c1
python eval.py --precision int8 --seed 1
runarmstop1
runs1fp32 and int8 · seed 10.7598 · 0.7569
runs3fp32 and int8 · seed 30.7599 · 0.7572

calibration set size: how many images INT8 needs

design not confirmed: rb confirm e2 lists it

Record

Runs

runs1 fp32 and int8 · seed 1
runruns1in e1

fp32 and int8 · seed 1

run · recorded 2026-10-12 09:14 UTC
produced 2026-10-12 09:02 (file)

fp32: top1=0.7598 onnx_top1=0.7569 · int8: top1=0.7569 onnx_top1=0.7566

counts toward c1, c2, c3 · made ckpt-int8-s1 · read imagenet-val

python eval.py --precision int8 --seed 1
runs2 fp32 and int8 · seed 2 · retracted
runruns2retracted

fp32 and int8 · seed 2

retracted 2026-10-13: the wrong checkpoint was loaded

replaced by runs2b

runs2b fp32 and int8 · seed 2
runruns2b

fp32 and int8 · seed 2

replaces runs2 · counts toward c1

runs3 fp32 and int8 · seed 3
runruns3

fp32 and int8 · seed 3

counts toward c1, c2

runsql7 fp32 and int8 · results/runs.db row 7
runrunsql7

fp32 and int8 · results/runs.db row 7

fp32: latency_ms=6.1 · int8: latency_ms=3.9

runc128 fp32 and int8 · calib_images 128
runrunc128

fp32 and int8 · calib_images 128

runc512 fp32 and int8 · calib_images 512
runrunc512

fp32 and int8 · calib_images 512

same bytes as what runs1 made

Analyses

a1 per-class bootstrap over images
analysisa1

per-class bootstrap over images

reads runs1, runs2b, runs3 · used by c4

Resources

imagenet-val · ImageNet-1k validation · evaluation data · remote
resourceimagenet-val

ImageNet-1k validation

read by runs1, runs3 and 6 more

ckpt-int8-s1 · ResNet-50 INT8, seed 1 · checkpoint · present
resourceckpt-int8-s1

ResNet-50 INT8, seed 1

made by runs1

python quantize.py --calib 512 --seed 1

History

  1. 2026-10-19 · accepted c1: within the one-point budget on three seeds
  2. 2026-10-13 · runs2 retracted: the wrong checkpoint was loaded

Your call (4)

4 calls wait on you.

Not reproduced · estimate 0.7597 outside 0.7613 ± 0.001fp32 top1 matches the published 76.13 c2
your callReview this resultc2

fp32 top1 matches the published 76.13

Why this needs a call

Not reproduced · estimate 0.7597 outside 0.7613 ± 0.001

It rests on 3 runs · 1 run not counted · the whole finding

Record a call

  • Accept
    rb decide c2 accept --why '‹why›'
    Its review will read: accepted ‹day›: ‹why›
  • Reject
    rb decide c2 reject --why '‹why›'
    Its review will read: rejected ‹day›: ‹why›
  • Investigate
    rb decide c2 investigate --why '‹why›'
    Its review will read: investigating since ‹day›: ‹why›
Gate missedcalibration set size: how many images INT8 needs e2
your calle2

calibration set size: how many images INT8 needs

Record a call

  • Continue
    rb decide e2 continue --why '‹why›'
  • Park
    rb decide e2 park --why '‹why›'
  • Stop
    rb decide e2 stop --why '‹why›'
Rests on one runthe ONNX export holds onnx_top1 c3
your callc3

the ONNX export holds onnx_top1

It rests on 1 run

What waits for your confirmation, listed apart →

About this record

What the words on this page mean
command (as given)
the command the run names, as given; rb did not run it
Retrospective
declared after its evidence existed