Keeping graphics processors fed, whether for large language models or the latest AAA games, takes a lot of memory bandwidth. That's exactly what Samsung's GDDR7 memory module, unveiled Wednesday, claims to deliver.

Today most GPUs and accelerators used in AI/ML work use speedy albeit expensive high-bandwidth memory that's closely if not directly coupled to the graphics processor silicon. In the consumer and workstation markets, graphics double data rate (GDDR) DRAM chips reign in all but the lowest budget cards.

With the introduction of 16-gigabit GDDR7 modules, Samsung is making some bold performance and efficiency claims. The manufacturing titan says the modules are capable of delivering up to 1.5 TB/sec of memory bandwidth - about 1.4x that of 24-gigabit GDDR6, which topped out at around 1.1 TB/sec. The chaebol says it's also boosted its per-pin speed to 32 Gb/sec.

According to Samsung, these improvements in performance efficiency are down to a shift from non-return-to-zero (NRZ) signaling to pulse amplitude modulation - more specifically PAM3. "PAM3 allows 50 percent more data to be transmitted than NRZ within the same signaling cycle," the biz claimed.

On top of the performance improvements, Samsung says it's also managed to drop power consumption by about 20 percent compared to GDDR6, and plans to offer a low voltage option for power constrained products - such as laptops and tablets.

All of this may sound impressive - but if our math is right, that 1.5TB/sec bandwidth claim relies on a fairly fat 384-bit memory interface. Unfortunately many chipmakers, including Nvidia and AMD, have taken to shrinking the memory bus over the past few generations. Nvidia halved the memory bus on its recently announced 4060 TI GPU from 256 bits and 448GB/sec of bandwidth to 128 bits and 288GB/sec.

Assuming memory buses on next-gen cards don't shrink further, the availability of GDDR7 should help to boost bandwidth and improve overall performance. But not as much as it's promising, unless the buses start getting bigger again.

However, it'll be a while before you can expect to see Samsung's GDDR7 modules in the wild. The Korean giant says the chips will start making their way to key customers later this year. Because of this, we don't expect to see GDDR7-toting GPUs until at least CES 2024 in January at the earliest - and likely only in high-end cards.

We should note that at the end of June Micron teased [PDF] its upcoming GDDR7 RAM too - due to arrive in the first half of 2024.

The launch comes as Samsung slogs through a weak memory market, which has battered the mega-corp's profit margins and forced creative inter-division cash infusions.

Earlier this month, Samsung warned that its second-quarter profits would likely fall 96 percent year over year. And while leading-edge chip families, like GDDR7, do command higher margins, the industry watchers at Trendforce remain doubtful that GPU demand from AI will be enough to reinvigorate the industry.

On the bright side, rumor has it Samsung has managed to work out the kinks in its 3nm and 4nm process nodes to achieve yields on par with or better than rival TSMC. ®

https://www.theregister.com//2023/07/20/samsung_gddr7_memory/