Does the LZ4 multithreaded scaling advantage hold beyond level -1?

Short answer: yes, at every level tested. After one build-script fix uncovered by this experiment, the optimized LZ4 binary beats stock LZ4 v1.10.0's built-in multithreading at all compression levels (-1, -3, -6, -9) and all multithreaded thread counts (8, 16, 32, and even 64 = 2× oversubscription), while producing byte-identical output.

Setup. Pristine upstream lz4 v1.10.0 (commit 0774d05, plain make) vs the optimized build (parallel chunk reads + PGO/LTO/native flags). Workload: lz4 -<level> -T<threads> -c silesia8.tar > /dev/null on a 1.7 GB corpus (8× silesia.tar) held in RAM (tmpfs), interleaved best-of-3 wall-clock runs on an idle 2× Xeon (16 physical cores / 32 hardware threads). Level -1 is LZ4's fast mode; levels -3/-6/-9 use the high-compression (HC) engine.

1 · Compression throughput: stock vs optimized (MB/s, best-of-3)

Level	Threads	Stock lz4 -T#	Optimized	Advantage	Stock scaling vs its own T1	Optimized scaling
-1 (fast)	1	426.6	419.3	0.98× (noise)	1.0×	1.0×
	8	1,614.9	3,028.0	1.88×	3.8×	7.2×
	16	1,583.3	3,670.3	2.32×	3.7×	8.8×
	32	1,519.4	3,810.5	2.51×	3.6×	9.1×
	64	1,473.2	3,785.0	2.57×	3.5×	9.0×
-3 (HC)	1	81.3	87.2	1.07×	1.0×	1.0×
	8	612.8	668.1	1.09×	7.5×	7.7×
	16	1,057.8	1,276.9	1.21×	13.0×	14.6×
	32	1,293.4	1,638.3	1.27×	15.9×	18.8×
	64	1,213.8	1,477.1	1.22×	14.9×	16.9×
-6 (HC)	1	49.5	53.0	1.07×	1.0×	1.0×
	8	384.7	414.2	1.08×	7.8×	7.8×
	16	730.9	798.0	1.09×	14.8×	15.1×
	32	929.6	1,068.5	1.15×	18.8×	20.2×
	64	883.6	1,021.5	1.16×	17.9×	19.3×
-9 (HC)	1	31.9	34.0	1.07×	1.0×	1.0×
	8	246.5	263.6	1.07×	7.7×	7.8×
	16	474.2	506.3	1.07×	14.9×	14.9×
	32	626.9	684.0	1.09×	19.7×	20.1×
	64	609.1	678.3	1.11×	19.1×	19.9×

2 · Throughput at 32 threads, level by level

Compression throughput at 32 threads, silesia×8 in tmpfs, best-of-3. The advantage is largest at -1 (I/O-loop-bound) and stays positive through -9 (compute-bound).

3 · Scaling curves: where stock stalls and where it doesn't

At level -1 stock is limited by its serial read→checksum→dispatch loop (scaling stalls at ~3.8×); the optimized parallel-read path reaches 9.1×. At level -9 compression itself is the bottleneck, so both scale to ~20× on 16 cores + hyperthreading — and the optimized build still wins on raw throughput.

4 · A finding this experiment uncovered (and fixed)

The first run showed the optimized binary losing ~5–6% to stock at HC levels, even single-threaded. To separate code from compiler flags, the optimized sources were rebuilt with plain make (identical flags to stock): that build won at every level (-3 T32: 1.19×, -6 T32: 1.09×, -9 T32: 1.03×, -9 T1: exact parity) — proving the deficit came from the build, not the code. Root cause: the PGO (profile-guided optimization) stage trained only on level-1 runs, so the compiler optimized the fast path at the HC path's expense. Fix (commit 032f1c0): build-optimized.sh now trains levels 1, 3 and 9. After retraining, the HC levels flipped from −6% to +7% single-threaded and +7–27% multithreaded, with level -1 unregressed.

5 · Honesty checks

Byte-identical output: at every level (-1/-3/-6/-9, T16) the optimized binary's compressed file has the same md5 as stock's — same frame format, same blocks, same checksums. Compression ratios are unchanged (2.10 / 2.60 / 2.70 / 2.72).
Cross-decode: the optimized binary's output, decompressed by the pristine stock binary, reproduces the original 1.7 GB corpus md5 exactly.
Single-thread parity: level -1 T1 checked with 5 extra interleaved runs (stock 3.92–4.07 s vs 4.03–4.14 s, ~1–2%, within noise); in-memory single-thread benchmarks unregressed after the PGO retrain.
Methodology: all comparisons interleaved best-of-3 on an idle machine, corpus in RAM, output discarded, identical command lines for both binaries.

Reproduce

git clone https://github.com/lz4/lz4 lz4-stock && git -C lz4-stock checkout 0774d05 && make -C lz4-stock -j
cd ~/LZ4-ks && ./build-optimized.sh                      # now trains PGO on levels 1, 3, 9
for i in $(seq 8); do cat ~/LZ4-ks-bench/silesia.tar >> /dev/shm/silesia8.tar; done
for L in 1 3 6 9; do for T in 8 16 32 64; do
  /usr/bin/time -f "stock -$L -T$T: %e s" ./lz4-stock/lz4 -$L -T$T -c /dev/shm/silesia8.tar > /dev/null
  /usr/bin/time -f "ours  -$L -T$T: %e s" ~/LZ4-ks/lz4   -$L -T$T -c /dev/shm/silesia8.tar > /dev/null
done; done

Raw measurement record: ~/LZ4-ks/OPTIMIZATION-RESULTS.md (Round 7, commit 032f1c0). Machine: 2× Intel Xeon 2.80 GHz, 16 physical cores / 32 hardware threads, lz4 v1.10.0 sources at upstream commit 0774d05.