News JEDEC finalizes HBM4 memory standard with major bandwidth and efficiency upgrades

[Admin]

HBM4 offers faster data rates, more channels, and higher memory capacities, with features like Directed Refresh and flexible voltage options to boost performance and reliability.

JEDEC finalizes HBM4 memory standard with major bandwidth and efficiency upgrades : Read more

 

[razor512]

Since managing multiple pools of VRAM has been around for a long time, e.g., the GTX 970 could manage 3 pools (3.5GB, 512MB, shared stem memory). With that in mind, would it be possible for a video card maker to do a hybrid approach, e.g., 16GB of HBM4, + 16GB of GDDR7?

 

[thestryker]

Since managing multiple pools of VRAM has been around for a long time, e.g., the GTX 970 could manage 3 pools (3.5GB, 512MB, shared stem memory). With that in mind, would it be possible for a video card maker to do a hybrid approach, e.g., 16GB of HBM4, + 16GB of GDDR7?

It would certainly be possible, but I'm fairly confident it would not make any sense monetarily. The memory controller would need to be optimized for both and HBM packaging makes things more expensive.

I'm not certain it would even make sense from a performance standpoint. The 50 series has significantly more memory bandwidth than the 40 series, but the performance improvement seems to largely mirror the core count and clock speed. This leads me to believe that the additional memory bandwidth is likely more related to the AI capabilities than raster/rt rendering.

 

[usertests]

I'm not certain it would even make sense from a performance standpoint. The 50 series has significantly more memory bandwidth than the 40 series, but the performance improvement seems to largely mirror the core count and clock speed. This leads me to believe that the additional memory bandwidth is likely more related to the AI capabilities than raster/rt rendering.

I think the main benefits for HBM on consumer GPUs would be power efficiency and less board space taken up, which is good for SFF. I'd be more interested to see HBM added as an L4 cache in consumer CPUs, or as a stack of VRAM for APUs.

But we won't get any of that until the cost and demand plummets, following at least a big AI bubble burst. Even outside of AI it's being used in Xeons, Epycs, supercomputer chips, etc.

 

[thestryker]

I think the main benefits for HBM on consumer GPUs would be power efficiency and less board space taken up, which is good for SFF.

I'm not really sure how much of a benefit it would be in practice though. You're absolutely right about it being more efficient and taking up less board space. The problem is most of the power consumption is the GPU and in turn the coolers required tend to be quite a bit larger than the PCB already.

I'd be more interested to see HBM added as an L4 cache in consumer CPUs, or as a stack of VRAM for APUs.

I'm not sure how much sense it'd make as a CPU cache due to latency, but it would make for a very interesting tiered memory system like Intel did with the Xeon Max. It would be fantastic for APU graphics though in something like an 8GB stack. Some future iteration of Strix Halo using two stacks instead of LPDDR would probably blow the doors off.

 

[Mr Majestyk]

More L2 cache would be desirable, rather than some super expensive memory not designed for desktop usage.

 

[DougMcC]

But we won't get any of that until the cost and demand plummets, following at least a big AI bubble burst. Even outside of AI it's being used in Xeons, Epycs, supercomputer chips, etc.

AI bust is years away at best. People in industry outside of the core ai circles are building like crazy, the demand for ai capacity is going to double or more every year for probably the next 5 years at least. My company has product plans out that far, and even existing models will clearly enable success. By the time we're 3 years out and next gen models offer even mild improvements, the list of projects will have grown rapidly.

There's no bust in sight.

 

[Thunder64]

AI bust is years away at best. People in industry outside of the core ai circles are building like crazy, the demand for ai capacity is going to double or more every year for probably the next 5 years at least. My company has product plans out that far, and even existing models will clearly enable success. By the time we're 3 years out and next gen models offer even mild improvements, the list of projects will have grown rapidly.

There's no bust in sight.

Just because you don't see it doesn't mean it doesn't exist. It's not imminent but many seem to agree there will be an AI bust.

 

[DougMcC]

Just because you don't see it doesn't mean it doesn't exist. It's not imminent but many seem to agree there will be an AI bust.

That's kind of the argument i'm making though. AI bust theorists think that there can be a sudden, large drop in AI investment. To think that requires blindness to the existence of the kinds of work and projects going on outside of the core AI providers. Just because they don't personally see that work, doesn't mean it doesn't exist. And I'm just saying: this exists. And its existence precludes the possibility of a short term (<5 years) major AI bust.