News AMD MI300X Guzzles Power, Rated for 750 Watts

[Admin]

AMD's latest product for the AI accelerator market, the MI300X, is a tour de force for the company's High Performance Computing (HPC) chops, but with great performance also come great power requirements.

AMD MI300X Guzzles Power, Rated for 750 Watts : Read more

 

[gg83]

Well 3x the transistor count equates to 3x the wattage. Compared to the 7900xtx.

 

[Metal Messiah.]

So the power requirements have gone up by 34-50% in a single generation. Doesn't come as a surprise though.

At the same time, this power increase was to be expected considering the design of the chip itself and the performance it has on offer. The chip delivers an 8x performance boost in AI workloads while being 5 times more efficient.

A recent Gigabyte Server roadmap also showcased how CPUs, GPUs & APUs are getting close to the 1KW power barrier. AMD certainly has the most power-hungry chip of all but the red team has also invested in a host of chiplet and packing technologies that allow them to significantly reduce power requirements.

Having chips stacked and packaged together also helps to reduce the relative Bits/Joule cost. So far, advanced packaging has alone provided a 50x reduction in communication power compared to when these chips were all stand-alone and put far to each other across the board.

Though efficiency is not a linear progression-like performance. Based on CPU & GPU efficiency trends, we are starting to see the progress flattening out so while achieving "Zettascale" performance in the next 10 years or so will be achievable, it will come at a significant efficiency cost.

 

[bit_user]

I see at least 2 trends driving this:

1. The more expensive new process nodes get per mm^2, the more incentive there's going to be to run it at higher clocks.
2. Chiplet technology has unlocked a path towards transistor counts increasing faster than process node density increases, which naturally means it's going to take more power to operate them.

 

[bit_user]

Based on CPU & GPU efficiency trends, we are starting to see the progress flattening out so while achieving "Zettascale" performance in the next 10 years or so will be achievable, it will come at a significant efficiency cost.

HPC facilities are ultimately somewhat constrained in how much power they can consume and how much heat they can dissipate. Lisa Su was clear about the need to find technological innovations to improve efficiency, since the default trajectory would consume far too much power. IIRC, the difference is like orders of magnitude.

 

[The Historical Fidelity]

AMD's latest product for the AI accelerator market, the MI300X, is a tour de force for the company's High Performance Computing (HPC) chops, but with great performance also come great power requirements.

AMD MI300X Guzzles Power, Rated for 750 Watts : Read more

Hi just want to point out an error in the article. In the comparison table, the memory type and memory bandwidth for the 7900xtx are wrong. It uses GDDR6 not GDDR5 and bandwidth is close to 1TB/s not 384GB/s

 

[prtskg]

Compared to previous gen, 250X, TDP has increased 50% while transistors has increased just north of 2.62 times. Let's see how performance increases.

 

[Kamen Rider Blade]

That 750 Watts is powering 8x GPU Chiplets shiped & 192 GiB of HBM3 on a single large Tile.

That's not half bad IMO.

 

[Metal Messiah.]

Hi just want to point out an error in the article. In the comparison table, the memory type and memory bandwidth for the 7900xtx are wrong. It uses GDDR6 not GDDR5 and bandwidth is close to 1TB/s not 384GB/s

+1 for this.

 

[gg83]

HPC facilities are ultimately somewhat constrained in how much power they can consume and how much heat they can dissipate. Lisa Su was clear about the need to find technological innovations to improve efficiency, since the default trajectory would consume far too much power. IIRC, the difference is like orders of magnitude.

I like the idea of turning the heat back to electricity to run the stuff like fans and led's. Who cares about power draw if none of it is wasted right?

 

[bit_user]

I like the idea of turning the heat back to electricity to run the stuff like fans and led's. Who cares about power draw if none of it is wasted right?

It's pretty difficult to recover energy from waste heat, or it'd be a lot more common. I'd be surprised if they could even achieve 5% efficiency.

The only way that waste heat isn't actually wasted is if you use it to heat a swimming pool or an eel farm, where you'd have needed some form of heating, regardless.

• https://www.tomshardware.com/news/japanese-data-center-starts-eel-farming-side-hustle
• https://www.tomshardware.com/news/heata-server-hot-water-trial-uk

 

[motocros1]

I like the idea of turning the heat back to electricity to run the stuff like fans and led's. Who cares about power draw if none of it is wasted right?

i believe turning water into steam would be the most effecient way to reclaim waste heat, and i hope the computer parts are not getting that hot.

 

[gg83]

It's pretty difficult to recover energy from waste heat, or it'd be a lot more common. I'd be surprised if they could even achieve 5% efficiency.

The only way that waste heat isn't actually wasted is if you use it to heat a swimming pool or an eel farm, where you'd have needed some form of heating, regardless.

• https://www.tomshardware.com/news/japanese-data-center-starts-eel-farming-side-hustle
• https://www.tomshardware.com/news/heata-server-hot-water-trial-uk

Sorry, I wasn't talking about current technology. Just like nanomachines for medical purposes, great idea just will be a while before it happens. I love that article about the eel farm.

 

[bit_user]

i believe turning water into steam would be the most effecient way to reclaim waste heat, and i hope the computer parts are not getting that hot.

This is an interesting point. As far as I know (which isn't much), the best method of extracting energy from heat is indeed using super-heated steam. While I agree with you that we don't want computer parts getting that hot, there's another trend in HPC that could theoretically dovetail with energy recovery: phase-change, immersion cooling.

In this case, they submerge hot components in a bath of non-conductive fluid with a lower boiling point than water.

​

Intel Looks to Immersion Cooling to Improve HPC Efficiency, Density

Intel aims to fix the relationship between electronics and liquids.

www.tomshardware.com

​

So far, I guess these baths are relatively open, for ease of serviceability. However, if they were sealed, you could harness the pressure differential, between the "vapor side" and the condenser. Two obvious questions are how much such a system would impact cooling efficiency and how much energy could you really recover by it?