Why GPUs want high-bandwidth reminiscence
With out HBM chips, an information middle server’s reminiscence system can be unable to maintain up with a high-performance processor, resembling a GPU, in response to Lee. HBMs are what provide GPUs with the info they course of. “Anybody who purchases a GPU for AI computation can even want high-bandwidth reminiscence,” Lee mentioned.
“In different phrases, high-performance GPUs can be poorly utilized and sometimes sit idle ready for information transfers. In abstract, excessive demand for SK Hynix reminiscence chips is brought on by excessive demand for Nvidia GPU chips and, to a lesser extent, related to demand for different AI chips resembling these from AMD, Intel, and others,” he mentioned.
“HBM is comparatively new and selecting up a robust momentum due to what HBM provides — extra bandwidth and capability,” mentioned Gartner analyst Gaurav Gupta. “It’s totally different than what Nvidia and Intel promote. Aside from SK Hynix, the state of affairs for HBM is comparable for different reminiscence gamers. For Nvidia, I consider there are constraints, however extra related to packaging capability for his or her chips with foundries.”
Whereas SK Hynix is reaching its provide limits, Samsung and Micron are ramping up HBM manufacturing and will be capable to help the demand because the market turns into extra distributed, in response to Lee.
The present HBM shortages are primarily within the packaging from TSMC (i.e., chip-on-wafer-on-substrate or CoWoS), which is the unique provider of the expertise. In accordance with Lee, TSMC is greater than doubling its SOIC capability and boosting capability for CoWoS by greater than 60%. “I count on the shortages to ease by the top of this 12 months,” he mentioned.
On the identical time, extra packaging and foundry suppliers are coming on-line and qualifying their expertise to help NVIDIA, AMD, Broadcom, Amazon, and others utilizing TSMC’s chip packaging expertise, in response to Lee.
