Skip to main content

Google Cloud TPU

Vendor: Google

Category: TPU Tensor Processing Unit

Architecture: TPU Matrix

Introduction​

Google's custom Tensor Processing Unit (TPU), delivering extreme acceleration for AI training and inference on frameworks such as TensorFlow/JAX/PyTorch. Current mainstream models include TPU v5e (inference-optimized), TPU v5p (training-optimized), and TPU v4.

Specifications​

ModelComputeMemoryInterfaceTDPProcess
v6e (Trillium)918 TFLOPS (BF16) / 1,836 (INT8)32GB HBM (per chip, 1,638 GB/s)ICI 800GB/s200W5nm
v5p459 TFLOPS (BF16)95GB HBM2e (per chip, 2,575 GB/s)ICI 1,200GB/s300W4nm
v4275 TFLOPS (BF16)32GB HBM2 (per chip)ICI Interconnect250W7nm

Official Website​

Visit Official Website

Driver Downloads​

Linux​

OS Support​

WindowsLinuxmacOSAndroid
❌✅ (GCP)❌❌

Version History​

VersionRelease DateDescription
TPU v6e2025Trillium architecture, doubled performance
TPU v5p2024Pod scaled to 8,960 chips
TPU v42023Native PyTorch/XLA support

Performance Benchmarks​

ModelTaskPerformance Metric
v6e Trillium PodGPT-3 175B Training~1.1 days (Google data)
v5p PodLlama 2 70B Inference~120 tok/s/chip
v5p PodJAX Large-Scale TrainingLinear scaling to thousands of chips
v4 PodMLPerf TrainingMultiple SOTA results

Pricing Information​

ModelReference PriceNotes
v6e Trillium~$4.20/chip/hOn-demand billing
v5p~$4.20/chip/hOn-demand billing
v4~$2.46/chip/hSpot pricing lower

Quick Setup​

GCP (gcloud CLI)​

# 1. Create TPU
gcloud compute tpus create tpu-node \
  --zone=us-central1-b \
  --accelerator-type=v5p-8 \
  --version=tpu-vm-v5-base

# 2. SSH Connection
gcloud compute tpus tpu-vm ssh tpu-node --zone=us-central1-b

# 3. Verify TPU
python3 -c "import jax; print(jax.devices())"

Code Examples​

Python (JAX on TPU)​

import jax
import jax.numpy as jnp

# Check TPU devices
print(f"TPU devices: {jax.devices()}")

# Auto-parallel matrix multiplication
x = jax.random.normal(jax.random.PRNGKey(0), (2048, 2048))
y = jnp.dot(x, x)
print(f"TPU matrix multiply: {y.shape}")

PyTorch/XLA​

import torch
import torch_xla
import torch_xla.core.xla_model as xm

# Use TPU backend
device = xm.xla_device()
x = torch.randn(1024, 1024, device=device)
y = torch.matmul(x, x)
print(f"TPU matrix multiply: {y.shape}")

Architecture Highlights​

  • TPU Matrix Architecture: Systolic arrays optimized for matrix operations, naturally suited for Transformer models
  • ICI Interconnect: Inter-chip interconnect bandwidth of 4.8Tbps (v5p), supporting linear scaling in large-scale Pods (thousands of chips)
  • Software Stack: JAX (native best) / PyTorch-XLA / TensorFlow — all supported

Model Compatibility​

Model/FrameworkSupport StatusNotes
JAX✅ Native BestGoogle preferred
PyTorch✅ XLA Backendtorch_xla
TensorFlow✅ NativeTPU native support
Llama / Qwen and other LLMs✅Both JAX/PyTorch supported
T5/BERT✅Google internal models native

Large-Scale Cluster Deployments​

Based on global AI supercomputing cluster statistics, Google Cloud TPU has accumulated over 94,856 chips deployed across 9 clusters in publicly disclosed deployments.

Chip Model Statistics​

Chip ModelTotal DeployedClusters
Google TPU v471,6804
Google TPU v5p8,9601
Google TPU v35,1202
Google TPU v15,0001
Google TPU v24,0961

Notable Deployment Clusters Top 10​

#Cluster NameTotal ChipsChip ModelOperator
1Google Oklahoma TPU v4 Pods32,768Google TPU v4 ×32,768Google, United States of America
2Gemini 1.0 Ultra training cluster A28,672Google TPU v4 ×28,672Google, United States of America
3Google Hypercomputer TPU v5p pod8,960Google TPU v5p ×8,960Google
4Paper on PaLM6,144Google TPU v4 ×6,144Google, United States of America
5Paper on AlphaZero5,000Google TPU v1 ×5,000Google, United States of America
6Google TPU v4 Pod4,096Google TPU v4 ×4,096Google, United States of America
7Google MLPerf 0.7 Submission4,096Google TPU v3 ×4,096Google, United States of America
8Google TensorFlow Research Cloud4,096Google TPU v2 ×4,096Google, United States of America
9Google TPUv3 POD Generic1,024Google TPU v3 ×1,024Google

If you are evaluating alternatives, the following products may also fit your scenario: