Rongchai Wang
Mar 25, 2026 11:36
NVIDIA details how Vera Rubin platform delivers 10x higher inference throughput per megawatt, reshaping AI data center economics and token factory revenue models.
NVIDIA published technical documentation claiming a staggering 1,000,000x improvement in inference throughput per megawatt across six generations of GPU architectures, positioning power efficiency as the critical metric for AI infrastructure economics.
The company’s framing is blunt: AI data centers are now “token factories” where revenue directly correlates with how efficiently power converts to billable AI output. With grid capacity increasingly constrained, operators can’t simply add more hardware—they need more intelligence per watt.
The Numbers Behind the Claims
According to NVIDIA’s technical breakdown, the upcoming Vera Rubin platform delivers up to 10x higher inference throughput per megawatt compared to current Blackwell systems, with proportionally lower token costs. For trillion-parameter workloads with high context windows, pairing Vera Rubin with NVIDIA’s Groq 3 LPX reportedly achieves 35x higher throughput per megawatt.
Blackwell Ultra GB300 NVL72 systems already show substantial gains over the previous Hopper generation—SemiAnalysis InferenceX data cited by NVIDIA indicates 50x higher throughput per megawatt and 35x lower token cost for running DeepSeek-R1.
The efficiency car analogy NVIDIA offers: if automotive fuel efficiency had improved at the same rate as their chips, one gallon would get you to the moon and back.
Where the Efficiency Comes From
NVIDIA attributes these gains to what they call “extreme co-design”—optimizing every layer from chip manufacturing through cooling systems to software orchestration.
On the manufacturing side, the cuLitho library accelerates mask synthesis by up to 70x, allowing a few hundred DGX systems to replace tens of thousands of CPU servers. Photomask cycles drop from two weeks to overnight runs using roughly one-ninth the power.
Cooling represents another major lever. Blackwell systems operate around 1.25 PUE with liquid cooling, while Vera Rubin moves to 100% liquid cooling at 1.1 PUE. The 45°C inlet water temperature allows ambient air cooling in many climates, reducing compressor runtime and shifting more power budget to actual compute.
At gigawatt scale, NVIDIA notes that up to 40% of power can be lost before reaching compute through cooling inefficiencies and overprovisioning. Their DSX orchestration system claims to address this, potentially allowing operators to run 30% more GPUs within the same power envelope.
The Revenue Calculation
NVIDIA frames AI inference as a tiered pricing model: free tiers for user acquisition, mid-tier for scale, and premium tiers with massive context windows commanding top-dollar per million tokens. Smarter models at higher context lengths generate more revenue.
For a one-gigawatt AI factory, the company claims Vera Rubin and Groq 3 LPX expand revenue per gigawatt by 10x compared to previous generations. Moving to next-generation hardware could yield 5x or more revenue for identical power consumption.
These claims carry obvious marketing weight—NVIDIA is selling hardware, after all. But the underlying economic logic holds: with power increasingly the binding constraint on AI infrastructure, operators who extract more tokens per megawatt capture more margin. Independent verification of these specific multipliers remains limited, though the directional trend toward efficiency-driven economics appears solid across the industry.
Image source: Shutterstock
