Skip to main content

NVIDIA GPU / CUDA

Vendor: NVIDIA

Category: GPU Graphics Processor

Architecture: Blackwell Ultra / Blackwell / Hopper / Ada Lovelace / Ampere / Volta / Pascal

Introduction​

NVIDIA GPU accelerated computing platform, covering the full lineup of GeForce/RTX gaming cards, RTX Professional workstation cards, and H100/B200 data center cards. All NVIDIA GPUs are unified through the CUDA platform for general-purpose computing acceleration.

Specifications​

Blackwell Ultra / Blackwell Architecture (present)​

ModelComputeMemoryBandwidthInterfaceTDPProcess
B300 SXM (Blackwell Ultra)9,000 TFLOPS (FP8) / 18,000 (FP4)270GB HBM3e7.7 TB/sSXM61,400WTSMC 4NP
B200 SXM4,500 TFLOPS (FP8) / 9,000 (FP4)192GB HBM3e8.0 TB/sSXM61,000WTSMC 4NP
B100 SXM3,500 TFLOPS (FP8) / 7,000 (FP4)192GB HBM3e8.0 TB/sSXM6700WTSMC 4NP
GB200 Superchip (2×B200+Grace)20,000 TFLOPS (FP8) / 40,000 (FP4)384GB HBM3e16 TB/sNVLink-C2C2,700WTSMC 4NP
GB300 Superchip (2×B300+Grace)40,000 TFLOPS (FP8) / 80,000 (FP4)540GB HBM3e15.4 TB/sNVLink-C2C2,800WTSMC 4NP

Hopper Architecture (present)​

ModelComputeMemoryBandwidthInterfaceTDPProcess
H200 SXM1,979 TFLOPS (FP8)141GB HBM3e4.8 TB/sSXM5700WTSMC 4N
H100 SXM51,979 TFLOPS (FP8) / 989 (FP16)80GB HBM33.35 TB/sSXM5700WTSMC 4N
H100 PCIe1,513 TFLOPS (FP8)80GB HBM32.0 TB/sPCIe 5.0350WTSMC 4N
H100 NVL (Dual)3,958 TFLOPS (FP8)2×80GB HBM32.0 TB/sNVLink700WTSMC 4N
H800 SXM5 (China-specific)1,979 TFLOPS (FP8)80GB HBM33.35 TB/sSXM5350WTSMC 4N

Ada Lovelace Architecture (present)​

ModelComputeMemoryBandwidthInterfaceTDPProcess
L40S362 TFLOPS (FP8) / 733 (INT8)48GB GDDR6 w/ECC864 GB/sPCIe 4.0300WTSMC 4N
L40362 TFLOPS (FP16) / 724 (INT8)48GB GDDR6864 GB/sPCIe 4.0300WTSMC 4N
L20119 TFLOPS (FP16) / 239 (INT8)48GB GDDR6864 GB/sPCIe 4.0275WTSMC 4N
L4242 TFLOPS (FP8) / 484 (INT8)24GB GDDR6300 GB/sPCIe 4.072WTSMC 4N
RTX 6000 Ada362 TFLOPS (FP8) / 733 (INT8)48GB GDDR6960 GB/sPCIe 4.0300WTSMC 4N

Ampere Architecture (present)​

ModelComputeMemoryBandwidthInterfaceTDPProcess
A100 SXM4 (80GB)312 TFLOPS (FP16) / 19.5 (FP32)80GB HBM2e2.0 TB/sSXM4400WTSMC 7N
A100 PCIe (80GB)312 TFLOPS (FP16) / 19.5 (FP32)80GB HBM2e2.0 TB/sPCIe 4.0250WTSMC 7N
A800 SXM4 (China-specific)312 TFLOPS (FP16)80GB HBM2e2.0 TB/sSXM4400WTSMC 7N
A40150 TFLOPS (FP16) / 37.4 (FP32)48GB GDDR6696 GB/sPCIe 4.0300WTSMC 7N
A30165 TFLOPS (FP16) / 10.3 (FP32)24GB HBM2e933 GB/sPCIe 4.0165WTSMC 7N
A10125 TFLOPS (FP16) / 31.2 (FP32)24GB GDDR6600 GB/sPCIe 4.0150WTSMC 7N
A16 (4×MIG)120 TFLOPS (FP16)4×16GB GDDR64×448 GB/sPCIe 4.0250WTSMC 7N

Volta / Pascal Architecture (present)​

ModelComputeMemoryBandwidthInterfaceTDPProcess
Tesla V100 SXM2 (32GB)125 TFLOPS (FP16) / 15.7 (FP32)32GB HBM2900 GB/sSXM2300WTSMC 12nm
Tesla V100 PCIe (16/32GB)125 TFLOPS (FP16) / 15.7 (FP32)16/32GB HBM2900 GB/sPCIe 3.0250WTSMC 12nm
Tesla T465 TFLOPS (FP16) / 8.1 (FP32)16GB GDDR6300 GB/sPCIe 3.070WTSMC 12nm
Tesla P100 (16GB)18.7 TFLOPS (FP16) / 10.6 (FP32)16GB HBM2720 GB/sSXM2/PCIe300WTSMC 16nm
Tesla P4047 TOPS (INT8) / 12 TFLOPS (FP32)24GB GDDR5X346 GB/sPCIe 3.0250WTSMC 16nm
Tesla P422 TOPS (INT8) / 5.5 TFLOPS (FP32)8GB GDDR5195 GB/sPCIe 3.075WTSMC 16nm

Official Website​

Visit Official Website

Driver Downloads​

Windows​

Linux​

macOS​

OS Support​

WindowsLinuxmacOSAndroid
✅✅⚠️ (AMD eGPU only)❌

Version History​

VersionRelease DateDescription
CUDA 12.82025-Q2Supports Blackwell architecture, B200/B100 full support
CUDA 12.42024-Q3Hopper performance optimization, H200 support
CUDA 12.02023-Q2H100/H200 full support, FP8 native support
CUDA 11.82022-Q4Ada Lovelace (L40S/L4) support
CUDA 11.02020-Q3Ampere (A100) support, MIG multi-instance
CUDA 10.02018-Q3Volta (V100) Tensor Core enhanced
CUDA 9.02017-Q3Volta V100 first support

Performance Benchmarks​

ModelTaskPerformance Metric
B200 × 8Llama 3 405B Training~2.5 days (estimated)
H100 SXM5 × 8GPT-3 175B Training~1.1 days (MLPerf)
H100 SXM5Llama 2 70B Inference~120 tok/s (FP16)
H200 SXM5Llama 3 70B Inference~140 tok/s (FP8)
A100 SXM4 × 8GPT-3 175B Training~3.5 days (MLPerf)
L40S × 4Whisper-large-v3~18× real-time transcription
L4 × 1Stable Diffusion XL~3.5s/img (batch=1)
Tesla T4 × 1BERT-large Inference~1,200 qps
RTX 4090Stable Diffusion XL~1.8s/img (batch=1)

Pricing​

ModelReference PriceNotes
B200 SXM$30,000-45,000Mass production in 2025
GB200 NVL$60,000-80,000Superchip (2 GPU + Grace CPU)
H100 SXM5$25,000-35,000Market price volatile due to supply
H200 SXM5$30,000-40,000HBM3e large memory version
H800 SXM5$15,000-20,000China-specific version
A100 80GB$10,000-15,000Substantial second-hand market
A800 80GB$8,000-12,000China-specific version
L40S$7,500-10,000Dual-purpose inference/graphics
L4$3,000-4,500Top choice for low-power inference
Tesla T4$2,000-3,000Entry-level inference card (lower used)
Tesla V100 32GB$2,500-4,000Discontinued, mainly used market

Quick Installation​

Linux (Ubuntu 22.04)​

# 1. Install NVIDIA driver
sudo apt update
sudo apt install nvidia-driver-550

# 2. Install CUDA Toolkit
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install -y cuda-toolkit-12-8

# 3. Verify installation
nvidia-smi
nvcc --version

Windows​

Download and install NVIDIA Game Ready Driver and CUDA Toolkit. Reboot and run nvidia-smi to verify.

Code Examples​

Python (PyTorch)​

import torch

# Check CUDA availability
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU: {torch.cuda.get_device_name(0)}")
print(f"CUDA version: {torch.cuda.version.cuda}")

# Simple tensor operation
x = torch.randn(1000, 1000).cuda()
y = torch.matmul(x, x)
print(f"GPU matrix multiply result shape: {y.shape}")

CUDA C (Hello World)​

#include <stdio.h>
__global__ void hello() { printf("Hello from GPU thread %d!\n", threadIdx.x); }
int main() {
    hello<<<1, 5>>>();
    cudaDeviceSynchronize();
    return 0;
}

Compile: nvcc hello.cu -o hello && ./hello

Architecture Highlights​

  • Blackwell Architecture (B200/B100): Dual-die design (208 billion transistors), 2nd-gen Transformer Engine, FP4 inference support; NVLink 5th gen 1.8TB/s interconnect; 10TB/s inter-chip interconnect unified as single GPU
  • Hopper Architecture (H100/H200): Introduced Transformer Engine with automatic FP8/FP16 switching; TMA (Tensor Memory Accelerator) asynchronous memory transfer; FP8 precision native support
  • Ada Lovelace Architecture (L40S/L4): 4th-gen Tensor Core, FP8 inference support; AV1 hardware-accelerated encoding; Omniverse/graphics rendering optimized
  • Ampere Architecture (A100/A40): 3rd-gen Tensor Core, TF32/BF16 native support; MIG multi-instance GPU virtualization; structured sparsity 2× acceleration
  • Volta Architecture (V100): First introduction of Tensor Core (FP16/FP32 mixed precision); 900GB/s HBM2 bandwidth
  • Software Stack: CUDA → cuDNN → cuBLAS → TensorRT → Triton (OpenAI), the most mature ecosystem

Model Compatibility​

Model/FrameworkSupportNotes
PyTorch✅ NativePreferred CUDA backend platform
TensorFlow✅ NativeFull GPU support
JAX✅ NativeCUDA backend
Llama / Qwen and similar LLMs✅vLLM / TensorRT-LLM / llama.cpp all supported
Stable Diffusion✅xFormers acceleration
Whisper✅Faster-Whisper (CTranslate2)

Large-Scale Cluster Deployments​

Based on global AI supercomputing cluster statistics, NVIDIA CUDA has accumulated over 1,620,688 chips deployed across 249 publicly disclosed clusters.

Chip Model Statistics​

Chip ModelTotal DeployedCluster Count
NVIDIA H100 SXM5 80GB948,79278
NVIDIA A100191,78273
NVIDIA H200 SXM178,8008
NVIDIA V10086,37635
NVIDIA GH20059,90811
NVIDIA Tesla V100 SXM251,99616
NVIDIA GB20030,0001
NVIDIA A100 SXM4 80 GB20,65212
NVIDIA A100 SXM4 40 GB13,49611
NVIDIA Tesla P100 PCIe 16GB8,7442
NVIDIA Tesla K40c8,3202
NVIDIA Tesla K20X7,2242
NVIDIA P1005,1543
NVIDIA Tesla P100 SXM22,1561
NVIDIA Tesla K801,7281
NVIDIA Tesla V100 DGXS 32 GB1,5361
NVIDIA Tesla K40m1,4721
NVIDIA Tesla V100 SXM2 32 GB1,0441
NVIDIA A100 PCIe4922
NVIDIA A40 PCIe4001
NVIDIA Quadro RTX 50003601
NVIDIA L402561

Notable Deployment Clusters Top 10​

#Cluster NameTotal ChipsChip ModelOperator
1xAI Colossus Memphis Phase 3230,000NVIDIA H100 SXM5 80GB ×200,000 + NVIDIA GB200 ×30,000xAI, United States of America
2xAI Colossus Memphis Phase 2200,000NVIDIA H100 SXM5 80GB ×150,000 + NVIDIA H200 SXM ×50,000xAI, United States of America
3xAI Colossus Memphis Phase 1100,000NVIDIA H100 SXM5 80GB ×100,000xAI, United States of America
4Meta 100k100,000NVIDIA H100 SXM5 80GB ×100,000Meta AI, United States of America
5OpenAI/Microsoft Goodyear Arizona100,000NVIDIA H100 SXM5 80GB ×100,000Microsoft,OpenAI, United States of America
6Oracle OCI Supercluster H200s65,536NVIDIA H200 SXM ×65,536Oracle, United States of America
7Tesla Cortex Phase 150,000NVIDIA H100 SXM5 80GB ×50,000Tesla, United States of America
8CoreWeave H200s42,000NVIDIA H200 SXM ×42,000CoreWeave, United States of America
9Oracle OCI Supercluster A100s32,768NVIDIA A100 ×32,768Oracle, United States of America
10Microsoft GPT-4 cluster25,000NVIDIA A100 ×25,000Microsoft,OpenAI, United States of America

If you're evaluating alternatives, the following products may also fit your scenario: