News
Nvidia high-speed NAND requirements for KV Cache may reach 100M IOPS
Wednesday, January 28, 2026 at 10:05 AM
Analysis of NVIDIA requirements for high-speed NAND/SSD storage to support KV Cache performance targets suggests IOPS requirements could reach 100M, translating to throughput between 100TB/s and 400TB/s depending on block size. Samsung data indicates the majority of these workloads are read-heavy with block sizes exceeding 1MB.
Context
Nvidia is reportedly targeting a massive performance benchmark of 100M IOPS for high-speed NAND and SSD integration within its KV Cache architecture. This shift indicates that future AI clusters will require unprecedented storage throughput to manage the expanding memory demands of large language models. As data centers transition toward LMCache solutions to improve efficiency, the industry is looking to Samsung and other memory leaders to provide hardware capable of handling these extreme, high-frequency read-heavy workloads.
The technical requirements for this transition are staggering, with potential bandwidth demands reaching between 100TB/s and 400TB/s based on block sizes ranging from 1MB to 4MB. A recent Samsung whitepaper highlights that 96% of I/O requests at the Linux block layer already exceed the 1MB threshold in these environments. This suggests a critical transition where traditional storage becomes a primary bottleneck for AI inference. Because these workloads are predominantly read-focused, the market is bracing for a surge in demand for specialized enterprise SSDs optimized for massive throughput.
Related Companies
SK Hynix
000660