NVIDIA's P100-to-Rubin gain: same decade, two numbers, one migrating bottleneck

The article's method is deliberately narrow: it uses only NVIDIA's official figures and re-normalizes them onto a single ruler, FP16 and dense. On that ruler, the 2016 P100 moves from 29 FLOPs/byte to 139 at V100 (2017), 153 at A100 (2020), and 295 at H100 (2022). That rise reflects Tensor Cores and lower-precision arithmetic, which the author says should be separated from gains in the silicon itself. In the 2024-to-2026 stretch, the picture flips: bandwidth catches up, and FLOPs/byte falls from 295 at H100 to 281 at B200 and 182 at Rubin.

The author also flags limits of the comparison. Rubin's memory bandwidth is listed as 22 TB/s in NVIDIA's developer blog and DGX Rubin NVL8 page, but a Vera Rubin NVL72 specification page shows 19.2 TB/s; the article adopts 22 TB/s and notes that the 19.2 TB/s figure would push Rubin's FP16 FLOPs/byte to about 208, without changing the direction of the trend. Power figures for Rubin were not found in NVIDIA's official materials, and the B200 1,000W value comes from a Lenovo document. The article's comparison to human history is presented as analogy only, not a causal claim.

Read this as a framework for reading launch numbers, not a verdict on any single chip. The stakes land on anyone sizing AI infrastructure or evaluating vendor claims, since a headline 'X×' figure can move by roughly an order of magnitude depending on precision and density assumptions, and capacity and power may shift where the next constraint sits.