When NVIDIA unveiled the Blackwell NVL72 architecture, the headline metric was a 4x reduction in training cost and a dramatic leap in inference throughput for massive Mixture-of-Experts (MoE) foundation models.
Our engineering team recently conducted the first comprehensive benchmark comparing an 8x H100 SXM5 cluster against an 8x B200 SXM node running the 405B parameter model under NovaPod's pre-tuned vLLM container environment.
By quantizing attention layers to native FP4 with per-tensor scaling factors, memory consumption dropped from ~810 GB to 425 GB, allowing the entire model to fit comfortably within the node's unified 1.5 TB memory space without cross-node network penalty.
Token decode throughput surged from 480 tok/s to over 1,220 tok/s while maintaining perplexity within 0.12% of baseline BF16 evaluations, proving the unmatched efficiency of Blackwell compute on NovaPod.