SK Hynix custom HBM boosts inference up to 5.15x

SK Hynix's proposal to embed compute functions into the base die of HBM marks a shift in how memory and processing are integrated. Traditional HBM focuses on bandwidth, but as LLM inference becomes increasingly limited by data movement, placing compute within the memory stack can reduce the need to shuttle large amounts of data between separate compute and memory chips. The 5.15x figure is presented as an upper bound, suggesting that actual gains will vary depending on the workload and system design.

At SEMICON Taiwan 2026, Senior Vice President and Fellow Hoshik Kim introduced this concept, signaling that SK Hynix is investing in architectures that blur the line between memory and computation. For business readers, this could lead to more efficient AI infrastructure, potentially lowering costs and energy consumption for running large models. However, the technology is still at the concept stage, and no production timeline or commercial availability was mentioned in the article.