JEDEC releases cheaper AI memory standard with 512-bit interface

JEDEC has published the SPHBM4 specification (JESD330-4), a new standard for high-bandwidth memory that aims to reduce the cost and manufacturing complexity of memory subsystems in AI processors. The acronym stands for Standard Package High Bandwidth Memory, and the specification combines HBM4 DRAM dies with a new, much narrower 512-bit interface paired with standard organic substrates—eliminating the need for expensive interposers and advanced packaging technologies like TSMC's CoWoS.

Conventional HBM3 and HBM4 memory use 1024-bit and 2048-bit interfaces respectively, delivering exceptional bandwidth but at significant cost. These wide interfaces consume substantial silicon area inside processors, demand expensive silicon interposers, and require advanced packaging with limited manufacturing capacity. SPHBM4 keeps the same HBM4 DRAM stacks as standard HBM4 but replaces the base die with a new SPHBM4 PHY/buffer die that implements a much narrower 512-bit external interface, allowing the memory to mount on inexpensive organic substrates. To preserve bandwidth despite the narrower interface, SPHBM4 supports dramatically higher data transfer rates ranging from 22.4 GT/s to 46.0 GT/s per pin.

The technical mechanism works by grouping HBM4's internal 32 memory channels (each 64 bits wide) into eight Quad Channels. Externally, each Quad Channel exposes 64 data pins (four groups of 16 bits), replacing the 256 data pins that four conventional HBM4 channels would require. To maintain bandwidth, these 64 pins operate at four times the data rate of the original HBM4 interface. The heart of SPHBM4 is the new base die's high-speed PHY, which maps each 16-bit external channel to four conventional 64-bit HBM4 channels. This introduces equalization, lane training, forward error correction (FEC), and other high-speed signaling features absent in HBM4's slower, parallel design. Each Quad Channel uses a shared command/address interface protected by FEC, while data transfers use dedicated differential clocks and ECC.

Capacity-wise, SPHBM4 can use stacks with 4, 8, 12, or 16 DRAM dies of 24 Gb or 32 Gb density, making the largest configuration a 64 GB memory stack built from sixteen 32 Gb dies—identical to HBM4E's maximum. The standard supports bump pitches greater than 90 µm and channel reaches up to 20 mm, both features that permit dropping the interposer and using cheaper organic substrate routing.

When comparing performance, one HBM4 stack delivers 2 TB/s at 8 GT/s, while HBM4E reaches 3–3.3 TB/s at 12–12.8 GT/s. By contrast, one SPHBM4 at its maximum 46 GT/s interface achieves 2.944 TB/s—substantial but falling short of HBM4E and likely to be surpassed by HBM4, HBM4E, and C-HBM4E in the foreseeable future. Latency is another concern: SPHBM4's sophisticated PHY inserts additional latency through serialization, deserialization, lane training, and FEC handling—a disadvantage for inference workloads that benefit from low latencies. Power efficiency also remains uncertain; while SPHBM4's fewer drivers and receivers could reduce I/O power consumption, the high-speed PHY itself is likely power-hungry, and without implementation details from DRAM makers, it is impossible to conclude which technology consumes less power overall.

An interesting secondary impact involves Chinese AI accelerator developers. Companies like Biren, Huawei, Moore Threads, and other entities blacklisted from TSMC services could benefit substantially from SPHBM4. A narrower interface helps pack more compute into chips made at trailing nodes without sacrificing memory bandwidth or capacity, and Chinese foundries do not offer CoWoS-equivalent packaging—so eliminating the interposer aligns with existing manufacturing infrastructure. However, SPHBM4 still requires HBM4 DRAM stacks, and only Samsung, SK hynix, and Micron can produce them; China's CXMT can barely manufacture HBM2E. Building a 46 GT/s PHY is also extremely challenging for Chinese IC developers, and assembling SPHBM4 on organic substrates is more viable only if local DRAM makers eventually develop competitive HBM4-class memory. Until that happens, the advantage for Chinese accelerator makers will remain incomplete.

In summary, SPHBM4 is a promising standard that may address a broader range of applications than HBM4 by lowering integration cost. However, HBM4, HBM4E, and C-HBM4E are expected to maintain performance leadership, making them the preferred choice for flagship AI accelerators in the near term.