NVIDIA H200 (Hopper Refresh)
Product Overview
The NVIDIA H200, released in 2024, is a memory-upgraded version of the H100. The key change is the upgrade from 80GB HBM3 to 141GB HBM3e, with bandwidth increasing from 3.35 TB/s to 4.8 TB/s (+43%). This delivers significant performance gains for 70B+ parameter model inference, enabling single-GPU loading of longer contexts.
Core Specifications
| Parameter | Value |
|---|
| Architecture | Hopper GH100 (refresh) |
| Process Node | TSMC 4N |
| Transistor Count | 80 billion |
| Memory | 141 GB HBM3e |
| Memory Bandwidth | 4.8 TB/s (4,800 GB/s) |
| CUDA Cores | 16,896 |
| Tensor Cores | 528 (4th Gen) |
| FP8 Tensor Core | 3,958 TFLOPS (sparse) |
| TDP | 700 W (SXM5) |
| Interconnect | NVLink 4.0 (900 GB/s), PCIe 5.0 |
| MIG | Up to 7 instances |
| Form Factor | SXM5 / PCIe 5.0 |
H100 vs H200 Comparison
| Metric | H100 | H200 | Improvement |
|---|
| Memory Capacity | 80 GB | 141 GB | +76% |
| Memory Type | HBM3 | HBM3e | Faster |
| Memory Bandwidth | 3.35 TB/s | 4.8 TB/s | +43% |
| FP8 Compute | 3,958 TFLOPS | 3,958 TFLOPS | Same |
| Price (Reference) | ~$25,000–30,000 | ~$30,000–35,000 | +20% |
Software & Drivers
Use Cases
- Large model inference (70B+ single GPU)
- Long-context (100K+ tokens) inference
- Large model fine-tuning
- HPC memory-bound workloads