TurboLoader Benchmark Dashboard

High-Performance ML Data Loading - Real-Time Performance Analytics

—
Peak img/s (cached)
—
vs PyTorch DataLoader
—
Metal resident img/s (consumed)
—
LLM tokens/s

Benchmark Configuration

Benchmark your own images

Drop in your own JPEG/PNG images and TurboLoader will decode → resize → normalize them and report real throughput (vs PyTorch). Requires the backend: python benchmark_server.py then open http://localhost:8000.

📁 Drop images here, or click to select
No images selected
Image throughput — Imagenette-160, img/s (real consumption)
Throughput vs PyTorch worker count

Only PyTorch uses per-worker processes, so only it scales with this knob. TurboLoader's fast path runs a single process-wide C++ thread pool sized to every core — it's already saturated at 1 worker (flat line). tf.data uses AUTOTUNE (manages its own parallelism), shown as a horizontal reference. PyTorch needs ~8 workers to approach what TurboLoader does with one internal pool.

LLM token streaming — tokens/s (TokenDataLoader vs numpy memmap)
GPU regimes — reference results (fixed hardware, see benchmark docs)

Static reference numbers measured on dedicated hardware — they don't re-run in this dashboard. Methodology, correctness checks, and the corrections history live in experiments/cuda/RESULTS.md, benchmarks/E2E_TRAINING_RESULTS.md and benchmarks/METAL_RESIDENT_RESULTS.md.

RegimeTurboLoaderBest alternativeHardware
On-the-fly (decode JPEGs every epoch) 28,527 img/sNVIDIA DALI 25,479 (+12%)RTX 3090
Pre-processed, fits in VRAM ~280k img/sFFCV-raw ~80k (3.5×)RTX 3090
Pre-processed, streaming > VRAM ~140k img/sFFCV-raw ~85k (1.6×)RTX 3090
Pre-processed, resident (unified memory) 757k produced / 433k consumed img/snumpy resident ~3.7kApple M4 Max
Pre-processed, CPU mmap (TBL-RAW, any hardware) 586k img/s raw serve (99k np.sum-consumed w/ prefetch)float32 RAM cache 137k at 3.3× the peak RSSApple M4 Max
End-to-end ResNet-18 training (epoch) TBL-RAW 3.64 s · TAR 3.76 sPyTorch DataLoader 3.92 s; pure-GPU floor 3.39 sRTX 3090
End-to-end video training (r3d_18, epoch) 4.19 s (1.16×)PyTorch DataLoader + PyAV 4.85 s; floor 2.11 sRTX 3090
LLM tokens delivered to device (32×1024) 168M tok/s (1.9×)nanoGPT get_batch 88MRTX 3090
Video: 1080p H.264 → 224px training batches (hardware decode) 2,556 frames/s (3.9×)OpenCV 657; PyAV 535; torchcodec 173Apple M4 Max
Video on NVIDIA: GPU-resident batches (cpu-decode backend) 371 frames/sOpenCV 312; PyAV 131; decord 604 leads on this weak-CPU hostRTX 3090 (WSL2)
End-to-end ResNet-18 training (median epoch) 4.37s (1.17×)PyTorch recipe 5.11s; pure-GPU floor 3.81sRTX 3090
End-to-end ResNet-18 on MPS (honest null result) 8.06s — a tie (1.00×)PyTorch 8.08s; MPS floor 7.61s (GPU-bound)Apple M4 Max

Key Features

SIMD-accelerated transforms
Resize, crop, normalize and more, vectorized with NEON / AVX2 / AVX-512. Half-pixel resize matching PIL/PyTorch/TF, with optional antialiasing.
FFCV/tf.data-style fast path
A persistent thread pool decodes → resizes → normalizes JPEGs directly into the output batch buffer in one pass, plus automatic libjpeg-turbo DCT scaled decode for large images.
Multi-modality
Images (JPEG/PNG/WebP in WebDataset TAR), LLM token streams (memory-mapped, next-token batches), and generic (N, …) arrays — one consistent, re-iterable DataLoader API.
Framework-ready output
Delivers NumPy / PyTorch (CHW) / TensorFlow (HWC) -ready batches, with a decoded cache, deterministic per-epoch shuffling, and DDP-safe distributed sharding.