Google unveils TPU 8t and TPU 8i at Cloud Next 26: eighth-generation AI chips split into dedicated training and inference designs

At Google Cloud Next '26 in Las Vegas on April 22, 2026, Google announced its eighth-generation Tensor Processing Units: the TPU 8t, built for massive-scale model training, and the TPU 8i, purpose-built for low-latency inference in agentic workloads. The chips end a decade of general-purpose TPU design in favor of a bifurcated architecture with one chip optimized for each phase of the AI development lifecycle.

What's new

TPU 8t — Training:

• Peak FP4 compute: 12.6 PFLOPs per chip
• HBM capacity: 216 GB, bandwidth: 6,528 GB/s
• Superpod scale: 9,600 chips, 2 PB shared memory
• Cluster scale: over 1 million chips, over 1.7K ExaFlops with near-linear scaling
• Virgo network: up to 47 petabits/sec of non-blocking bi-sectional bandwidth
• Price-performance: 2.7x improvement vs. Ironwood for large-scale training
• New: native FP4 (doubles MXU throughput), SparseCore for embedding lookups, TPUDirect RDMA, 10x faster storage access

TPU 8i — Inference:

• Peak FP4 compute: 10.1 PFLOPs per chip
• HBM capacity: 288 GB, bandwidth: 8,601 GB/s
• On-chip SRAM: 384 MB (3x previous generation) — KV caches fit entirely on-silicon
• Pod scale: 1,024 chips via new Boardfly topology
• Boardfly: 7 hops maximum vs. 16 in 3D torus — 56% reduction in network diameter, up to 50% lower communication-intensive latency
• Collectives Acceleration Engine (CAE): 5x reduction in on-chip collective latency
• Price-performance: 80% better inference price-performance vs. Ironwood
• Both chips: 2x better performance-per-watt

Native PyTorch support for TPUs is in preview. Existing JAX, PyTorch, and Keras code from Ironwood scales without modification.

Context

The TPU 8 generation marks a deliberate architectural split. Previous TPU generations were general-purpose accelerators. By creating TPU 8t and TPU 8i as separate products, Google acknowledges that training and inference workloads have diverged: training demands raw throughput and scale; inference demands low-latency inter-chip communication and large KV caches for concurrent agentic sessions.

The Boardfly topology on TPU 8i is drawn from Dragonfly network design principles. High-radix optical long-haul links between chip groups flatten the network and cut the diameter from 16 hops (3D torus configuration for a 1,024-chip pod) to 7 — a significant reduction for the token-generation bottleneck in long-context models.

Google's design philosophy, per the technical team: "At Google, our TPU design philosophy has always been centered on three pillars: scalability, reliability, and efficiency."

Why it matters

TPU 8t and TPU 8i represent the most capable non-NVIDIA AI chips available from a major cloud provider. Training price-performance of 2.7x and inference price-performance of 80% better than the previous generation mean Google's internal model training and serving costs drop sharply — while the same silicon now underpins the company's external cloud AI business.

The TPU 8i's 384 MB on-chip SRAM and sub-7-hop Boardfly network directly address the requirements of the next generation of long-context, multi-agent deployments. For AI teams evaluating alternatives to NVIDIA's H100/GB200 supply chain, TPU 8t/8i are the first technically comparable option from a hyperscaler with a public infrastructure offering — backed by the May 2026 Blackstone JV that will bring 500 MW of TPU capacity to the external market by 2027.