NovaPod
The cloud GPU market in late 2026 is undergoing its most significant architectural inflection since the transition from Ampere to Hopper: the commercial rollout of NVIDIA Blackwell B200 accelerators alongside widespread H200 141GB deployments.
For engineering leaders and ML infrastructure architects, evaluating these silicon tiers is no longer a simple question of raw hourly price. The convergence of native FP4 quantization, expanded HBM3e bandwidth (up to 8.0 TB/s on B200), and dynamic Key-Value (KV) cache consumption has decoupled the nominal dollar-per-hour metric from the true operational cost: cost per million generated tokens.
In this benchmark report, we analyze empirical throughput data collected across NovaPod's bare-metal clusters, comparing the NVIDIA B200 SXM, H200 SXM5 (141GB), H100 SXM5 (80GB), and consumer RTX 5090 / 4090 instances across serving, training, and fine-tuning workloads.
Architecture Breakdown: Blackwell vs Hopper Silicon Generations
The architectural leap from Hopper (GH100) to Blackwell (GB200/B200) centers on two foundational advances: dual-die packaging with a 10 TB/s chip-to-chip interconnect, and the second-generation Transformer Engine supporting microscopic scaling FP4 (4-bit floating point) arithmetic.
While Hopper pioneered FP8 mixed-precision training and inference, Blackwell natively implements FP4 tensor cores that double dense compute throughput while halving memory footprint. Paired with 192GB of HBM3e memory operating at 8.0 TB/s (compared to 3.35 TB/s on H100 SXM5 and 4.8 TB/s on H200), memory-bound decoding phases experience unprecedented acceleration.
Meanwhile, the H200 SXM5 serves as Hopper's ultimate iteration, equipping the GH100 die with 141GB of 4.8 TB/s HBM3e. For large context windows where model weights easily fit into memory but multi-turn KV caches explode, H200 delivers 1.76x the addressable VRAM of an H100 without requiring tensor-parallel multi-GPU sharding.
2026 Cloud GPU Hourly Spot Pricing & Specs Comparison
Below is an overview of active cloud spot marketplace rates, physical memory configurations, and peak floating-point throughput across leading accelerator classes available on NovaPod.
| Accelerator | VRAM (Type) | Memory Bandwidth | Peak Tensor FLOPS | Live Spot Range ($/hr) | Optimal Workload |
|---|---|---|---|---|---|
| NVIDIA B200 SXM | 192 GB HBM3e | 8,000 GB/s (8.0 TB/s) | 4,500 TFLOPS (FP4) / 2,250 TFLOPS (FP8) | $3.80 – $4.60 / hr | High-throughput 70B+ inference, FP4 MoE serving |
| NVIDIA H200 SXM5 | 141 GB HBM3e | 4,800 GB/s (4.8 TB/s) | 1,979 TFLOPS (FP8) | $2.60 – $3.20 / hr | 128k+ context LLM inference, DeepSeek 70B single-GPU |
| NVIDIA H100 SXM5 | 80 GB HBM3 | 3,350 GB/s (3.35 TB/s) | 1,979 TFLOPS (FP8) | $2.10 – $2.60 / hr | Standard FP8 serving, distributed pre-training runs |
| NVIDIA L40S | 48 GB GDDR6 | 864 GB/s | 733 TFLOPS (FP8) | $0.75 – $0.95 / hr | Agent sandboxes, multi-turn tool calling, embeddings |
| NVIDIA RTX 5090 | 32 GB GDDR7 | 1,792 GB/s (1.79 TB/s) | 820 TFLOPS (FP4/FP8) | $0.55 – $0.75 / hr | LoRA fine-tuning, ComfyUI, 14B-32B local inference |
| NVIDIA RTX 4090 | 24 GB GDDR6X | 1,008 GB/s (1.0 TB/s) | 330 TFLOPS (FP8) | $0.35 – $0.45 / hr | Rapid prototyping, batch image gen, 8B models |
Throughput Benchmarks: Token Generation on Llama 3.3 70B & Qwen 2.5
To quantify real-world throughput, we benchmarked vLLM 0.7.x and SGLang running Llama-3.3-70B-Instruct under concurrency loads ranging from 1 to 128 streams.
On an 8x H100 SXM5 cluster running FP8 weights, the cluster delivered an aggregate 1,840 tokens per second across 64 concurrent streams with an average Time-to-First-Token (TTFT) of 185ms.
On an 8x B200 SXM node running native FP4 weights with FP8 KV cache, the same cluster achieved 7,820 tokens per second under 128 concurrent streams—a 4.25x throughput multiplier. Crucially, the cost per million tokens dropped from $0.098 on H100 to $0.042 on B200.
For single-node deployments, an 8x H200 SXM5 node was able to serve Llama-3.3-70B with an active 128k context window across 32 concurrent sessions without offloading KV cache to host RAM, whereas the 80GB H100 cluster exhausted VRAM at 48k context under identical concurrency.
When to Choose H200 Over B200: Memory vs Compute Bottlenecks
Despite Blackwell's superior raw throughput, the H200 SXM5 remains the strategic choice for specific engineering scenarios:
1. Memory-Capacity-Bound Low-Concurrency Services: If your production traffic experiences sporadic bursts or low concurrency (< 8 streams) on long-context documents (e.g. legal analysis, codebase indexing), the workload is bounded by VRAM capacity rather than compute FLOPS. Paying for B200 compute capability that sits idle between decode steps is suboptimal compared to H200's lower hourly rate.
2. FP8/FP16 Compliance Constraints: In regulated domains where validation frameworks strictly require FP8 or BF16 precision without FP4 quantization loss, H200's 141GB memory allows running larger parameter sets at higher precision within a single node.
3. Immediate Bare-Metal Availability: While Blackwell clusters experience hyper-scaler waitlists, NovaPod's decentralized marketplace offers instantly launchable H200 instances with zero reservation lock-in.
The Economics of Per-Second Crypto Billing vs 1-Year Cloud Commitments
Traditional cloud providers force research teams into 1-year or 3-year take-or-pay reservation contracts to unlock reasonable H100/B200 hourly rates ($2.50-$3.00/hr). However, because AI development pipelines are inherently bursty—alternating between intensive multi-day pre-training runs and prolonged evaluation intervals—effective GPU utilization in enterprise clouds rarely exceeds 55-65%.
When accounting for unutilized idle time, an AWS/Azure 1-year reserved instance effectively costs $3.80 to $4.60 per active hour of compute.
NovaPod eliminates this deadweight loss through per-second streaming crypto settlement in USDT, USDC, and SOL. When your training run converges or evaluation completes, you terminate the pod immediately. Zero commitments, zero idle overhead, and instant hardware elasticity.
np pod create \
--accelerator B200_SXM \
--quantity 8 \
--template vllm-fp4 \
--model meta-llama/Llama-3.3-70B-Instruct \
--context-len 65536 \
--settlement crypto_usdt