Jon versus Dioptrin with a changing mask every frame
Apple M4 Pro, November5,2026; identical real Eiger9M sample and settings. Primary threshold230 masks an additional0.977% of otherwise valid pixels on average. Every frame includes new mask generation, normalization and the backend's required preparation.
| Implementation | Intensity frames/s | With Poisson errors frames/s |
|---|
| Jon, rebuild signal matrix | 15.0 | — |
| Jon, stacked signal/Poisson matrix | — | 3.0 |
| Jon signal + SciPy Poisson | 15.0 | 13.9 |
| Dioptrin0.7 public API control | 102.3 | 95.9 |
| Native prototype: current_csr_two_pass | 113.7 | 113.1 |
| Native prototype: fused_csr | 150.8 | 149.1 |
| Native prototype: csc_four_histograms | 254.4 | 206.3 |
The unmodified Dioptrin public-API control is 6.84× faster for intensity and 6.92× faster than the cheaper Jon+SciPy error adapter. Jon uses float32 weights; Dioptrin uses float64. This is the public mask-rebuilding path, not an intrinsic speed limit of Jon's C kernel.
Matrix preparation dominates Jon's cost because masks are baked into the prepared matrix. A changing mask repeats validation/copying, mask folding, analysis and layout construction. Stacking signal/variance rows further increases this cost; using SciPy for variance avoids much of that penalty.
All numerical checks passed. Actual Jon-versus-Dioptrin intensity relative L2 difference: 5.29e-10; SciPy Poisson sigma difference: 1.18e-15. The small intensity difference is consistent with Jon's float32 coefficient storage.
Protocol and limits
- input: Same 16 immutable bottom-up float64 Eiger9M frames, same float64 geometry export, 2000 uniform two-theta degree bins, full pixel splitting, solid-angle correction and polarization0.965. No IO or decompression timed.
- mask: New upper-threshold mask OR static mask every frame, from actual intensities. Thresholds323,230,207 match previous 0.098%,0.977%,4.709% additional masked pixels; frame counts asserted equal to previous native experiment.
- jon: Unmodified pinned build94264db5e0f812aaa71b91af286f560b00efb368, public chunk2sparseCSC constructor rebuilt every frame, explicit best previously measured CSC dot/dense route. Cache only original geometry CSC; original CSR64 retained to recompute denominator every frame. New masks, validation/casting, folding, matrix analysis/layout, allocations/workspace, native calculation, normalization and cleanup timed.
- precision: Jon float32 weights, float64 pixel values/output. Denominator recomputed from original float64 weights for both implementations. Stacked Poisson rows square canonical float64 weights before conversion tofloat32. SciPy Poisson adapter retains squared float64 CSR geometry and recomputes variance every frame.
- errors: Stacked adapter rebuilds combined signal/variance CSC each frame. Lower-cost adapter rebuilds signal CSC only, and computes Poisson variance with fixed geometry CSR64 and changing masked signal in SciPy; all work timed.
- dioptrin_api: Unmodified0.7.0 public set_mask + integrate1d, same NumPy threshold generation. Eight independent processes with one Rayon thread each, retained per-worker geometry/LUT; startup and first LUT build excluded. Separate processes allow independent concurrent per-frame masks without single-image Python GIL serialization. Includes mask copy, cache invalidation, normalization, output materialization and result IPC.
- native_repeat: Fresh run of prior native experiment at8threads, threshold230, 5>=1s warm blocks. Native threshold mask generation/copy included, Python excluded. Exploratory CSC/fusedCSR are standalone prototypes, not shipped code.
- sampling: Jon five timed16-frame batches per threshold/error at8workers, three at1worker. One full warm/reference batch before each workload. No per-frame masks/plans/denominators reused; all samples retained. DioptrinAPI five timed16-frame batches pererror atthreshold230.
- validation: Every warm frame checked against canonical float64 CSR and against rounded-weight matrix. Actual DioptrinAPI outputs checked against Jon+SciPy outputs. Mask counts match prior experiment exactly.
- limits: Measures cached detector frames with full per-frame mask/table/normalization updates. Not full GUI, compressed-file throughput, cold-cache, detector geometry rebuild, or other dynamic masking algorithms. Different parallel worker models and weight precision explicitly reported. Public API rebuilding performance does not limit a future native dynamic-mask kernel.
Artifacts
All results and ranges · Conclusions · Full provenance · Jon benchmark · DioptrinAPI control
Reproduce sequentially using the recorded temporary environment: run.py, then run_controls.py. The large frame and LUT buffers remain at their existing /tmp paths. Source and input hashes are recorded. No Dioptas or production Dioptrin code was changed for these measurements.