Google explores replacing HBM with external DRAM memory pools for TPU v8 using photonic interconnects

Google is reportedly developing a new architecture for its TPU v8 that replaces HBM with independent DRAM memory cabinets connected via photonic technology. This approach addresses HBM capacity shortages and the physical space limitations of motherboard-fixed HBM. The system uses all-optical interconnects via Optical Circuit Switching (OCS) and customized CXL protocols, potentially doubling memory capacity per TPU while reducing reliance on TSMC's CoWoS packaging by freeing up silicon interposer area for compute cores. Lightmatter is identified as a primary supplier for the photonic packaging interfaces.