News
Nvidia CEO Jensen Huang says Groq hardware could account for 25% of data center infrastructure
Tuesday, March 17, 2026 at 12:22 AM
Groq is scaling its existing architecture to produce chips featuring 500 MB of SRAM. During a recent event, Nvidia CEO Jensen Huang reportedly noted that up to 25% of data centers could eventually be composed of Groq's hardware for specialized AI inference workloads.
Context
At the GTC 2026 conference on March 16, 2026, Nvidia CEO Jensen Huang officially integrated the Groq 3 LPU into the company’s next-generation Vera Rubin platform. During the keynote, Huang highlighted the shifting landscape of the AI market from training to real-time inference, specifically identifying Groq's architecture as a critical component. Huang stated that Groq hardware could account for up to 25% of a modern data center’s infrastructure, positioning the technology as a primary engine for ultra-low-latency token generation in agentic AI. This statement follows Nvidia’s strategic $20 billion acquisition of Groq, aimed at defending its leadership against rivals like AMD and Broadcom. The integration pairs Nvidia's Rubin GPUs with Groq 3 LPX racks, which house 256 chips and deliver a massive 40 PB/s of SRAM bandwidth. These systems are designed to power trillion-parameter models, with Huang predicting a $1 trillion order backlog by the end of the year as the industry enters this 'inference inflection' phase. The Groq-powered racks are expected to be available in the second half of 2026.
Sources (12)
NVIDIA Vera Rubin Opens Agentic AI Frontier | NVIDIA NewsroomInside NVIDIA Groq 3 LPX: The Low-Latency Inference Accelerator for the NVIDIA Vera Rubin Platform | NVIDIA Technical BlogHow to watch Jensen Huang’s Nvidia GTC 2026 keynote — and what to expect | TechCrunchNVIDIA GTC 2026 Keynote Live Coverage - ServeTheHomeGroq: Nvidia's $20 Billion Bet on AI Inference - EE TimesNVIDIA GTC 2026: Rubin GPUs, Groq LPUs, Vera CPUs, and What NVIDIA Is Building for Trillion-Parameter Inference - StorageReview.comNvidia Puts Groq LPU, Vera CPU And Bluefield-4 DPU Into New Data Center RacksAdvancing the American AI Stack | Groq is fast, low cost inference.
Related Companies
Nvidia
NVDA