Nvidia reveals Rubin and LPX heterogeneous architecture to challenge AI accelerator startups

Nvidia has introduced a new heterogeneous inference architecture featuring the Rubin GPU and LPX (LPU) accelerator to compete with AI startup specialized architectures. This design offloads the MLP phase of model decoding to the LPX's 128GB SRAM while keeping the memory-intensive Attention phase (KV cache) on the GPU's HBM. This configuration enables peak throughput exceeding 1,000 tokens per second, a 2x improvement over the Blackwell-based NVL72. The shift aims to close the latency and speed gaps previously exploited by startups like Cerebras, d-Matrix, and MatX. Additionally, Nvidia confirmed a speculative decoding workflow where the LPX generates draft tokens and Rubin GPUs verify them.