Google Built Two AI Chips — Here’s Why Nvidia Should Care

For years, every major cloud provider has poured billions into custom silicon while continuing to write enormous checks to Nvidia. Google just made the boldest move yet. At Cloud Next 2026 on April 22, the company split its eighth-generation Tensor Processing Unit into two entirely separate chips — one optimized purely for training, the other for inference — signaling that the era of one-size-fits-all AI hardware is over. The timing is deliberate: as agentic AI workloads explode, the performance demands for training a model and serving it in real time are diverging so fast that a single chip architecture can no longer do both well.

The Split Strategy: Why Two Chips Beat One

TPU 8t: Built for Massive-Scale Training

The training-focused TPU 8t uses a 3D torus network topology and scales up to 9,600 chips in a single pod — up from 9,216 with Ironwood. But the real number that matters is the cluster ceiling: Google says TPU 8t supports near-linear scaling up to one million chips, delivering a combined 121 ExaFlops of compute. That is the kind of capacity needed to train frontier models that now routinely consume tens of thousands of GPU-hours. The chip also introduces native four-bit floating point math and a SparseCore accelerator for irregular memory access patterns, doubling throughput while maintaining accuracy with smaller memory footprints. Overall, Google claims a 2.7x performance-per-dollar improvement over Ironwood for large-scale training jobs.

TPU 8i: Purpose-Built for Inference at Scale

Inference is an entirely different animal. As mixture-of-experts models grow and chain-of-thought reasoning chains lengthen, the bottleneck shifts from raw compute to memory bandwidth and latency. TPU 8i addresses this head-on with 288 GB of high-bandwidth memory paired with 384 MB of on-chip SRAM — triple the previous generation. A new Collectives Acceleration Engine reduces on-chip latency by up to 5x, while a custom Boardfly ICI network topology connects up to 1,152 chips and cuts all-to-all communication hops by 50%. The result: roughly 80% better performance-per-dollar versus Ironwood at low-latency targets, enabling Google to serve nearly twice the customer volume at the same cost.

The Nvidia Question: Supplement, Not Replace

Despite the aggressive specs, Google is careful not to frame TPU 8 as an Nvidia killer. At the same event, the company confirmed it will offer Nvidia’s upcoming Vera Rubin chip to Cloud customers later in 2026. Google and Nvidia are also collaborating on Falcon, an open-source software-based networking technology contributed to the Open Compute Project. Analyst Patrick Moorhead pointed out that despite years of predictions about custom silicon displacing Nvidia, the GPU maker has only grown — now approaching a $5 trillion market cap.

The real dynamic is more nuanced. Google uses TPUs internally for its own models — Gemini, Veo, Imagen — while offering both TPUs and Nvidia GPUs to external customers. The split architecture lets Google optimize its own workloads aggressively while giving customers flexibility. For enterprises locked into Nvidia’s CUDA ecosystem, TPUs remain a harder sell. But for cost-sensitive inference workloads where Google controls the software stack, the 80% price-performance advantage is hard to ignore.

The Agentic Era Demands Specialized Hardware

Google explicitly positioned TPU 8 for what it calls “the agentic era” — a world where AI systems do not just answer questions but plan, reason, and take multi-step actions autonomously. This matters because agentic workloads create fundamentally different hardware demands. A single user query to an agentic system might trigger dozens of inference calls as the model reasons through sub-tasks, checks its work, and coordinates with other agents. That means inference volume per user interaction could increase by an order of magnitude compared to simple chatbot queries.

TPU 8i’s Collectives Acceleration Engine is specifically designed for this pattern — autoregressive decoding and chain-of-thought reasoning where the chip must rapidly process sequential token generation while maintaining coherence across a distributed cluster. The 19.2 Tb/s interconnect bandwidth is tuned for mixture-of-experts models where different parts of the network activate on different chips. This is not a general-purpose improvement; it is a targeted bet on the architecture of the next generation of AI applications.

What This Means for the AI Infrastructure Race

Google’s two-chip strategy arrives in a market where the infrastructure arms race shows no signs of slowing. Anthropic just committed $100 billion in cloud spending to AWS over the next decade. OpenAI’s $122 billion raise in Q1 2026 was partly justified by infrastructure needs. Microsoft is building custom chips (Maia) while maintaining its Nvidia partnership. Amazon has Trainium and Inferentia. Every hyperscaler is running the same playbook — build custom silicon for internal efficiency while offering Nvidia for external customers.

But Google may have the most vertically integrated stack of them all. It designs the chips, builds the models, controls the cloud platform, and operates the end-user products (Search, Workspace, Android) where those models are deployed. Both TPU 8 variants deliver 2x better performance-per-watt through integrated power management and liquid cooling — a detail that matters when data centers are increasingly constrained by power availability rather than floor space. With general availability expected later in 2026, the real test will be whether third-party developers adopt TPUs at scale or remain in Nvidia’s orbit.

Related

• Google Cloud launches two new AI chips to compete with Nvidia — TechCrunch
• Our eighth generation TPUs: two chips for the agentic era — Google Blog
• Google unveils new TPUs to power the next wave of AI training and inference — SiliconANGLE
• Google presents TPU 8t and 8i; splits training and inference — Techzine
• Google launches Ironwood TPU and previews eighth-gen split — The Next Web

Sources

1. TechCrunch — Google Cloud launches two new AI chips to compete with Nvidia (April 22, 2026)
2. Google Blog — Our eighth generation TPUs: two chips for the agentic era (April 22, 2026)
3. SiliconANGLE — Google unveils new TPUs to power next wave of AI training and inference (April 22, 2026)
4. Bloomberg — Google Cloud unveils new TPU chips to accelerate AI training and inference (April 22, 2026)