News Nvidia aims to solve AI's water consumption problems with direct-to-chip cooling — claims 300X improvement with closed-loop systems

[Admin]

Nvidia's GB200 NVL72 and GB300 NVL72 AI systems achieve 25 times greater energy efficiency and 300 times better water efficiency by using closed-loop direct-to-chip liquid cooling, at the cost of expensive data center redesigns.

Nvidia aims to solve AI's water consumption problems with direct-to-chip cooling — claims 300X improvement with closed-loop systems : Read more

 

[Pierce2623]

When are people other than Nvidia, OpenAI and a chosen few cloud providers supposed to start actually getting revenue out of AI?

 

[frogr]

"...the company has invested in reference designs of sealed liquid cooling solutions to avoid the use of evaporative liquid cooling solutions, in an effort to preserve water"

Open-loop evaporative cooling is not used at the chip level or rack level. It is used to produce cold water at the building level which is used to cool the air or water supplied to the server racks. The chilling technology at the chip/ rack level and building level are basically independent.

https://datacenters.microsoft.com/w...ure_Modern-Datacenter-Cooling_Infographic.pdf

 

[DS426]

"...the company has invested in reference designs of sealed liquid cooling solutions to avoid the use of evaporative liquid cooling solutions, in an effort to preserve water"

Open-loop evaporative cooling is not used at the chip level or rack level. It is used to produce cold water at the building level which is used to cool the air or water supplied to the server racks. The chilling technology at the chip/ rack level and building level are basically independent.

https://datacenters.microsoft.com/w...ure_Modern-Datacenter-Cooling_Infographic.pdf

Yeah, datacenters still have to expel the heat from the room (and I assume hot and cold aisles are still used). The only thing CLC provides is more power density. This in turn reduces serviceability of these racks.

As for these proprietary racks, DC operators aren't having a good time having to go to nVidia when there is almost any kind of problem, whether hardware or software related. This mini-monopoly hardware system is a step backwards. THN had an article on Pat G. at Intel complaining about the industry needing to get away from CUDA and instead embrace open standards, and yet here we are today with nVidia supposedly having up to 98% of AI training GPU market share. *facepalm*

 

[AngelusF]

THN had an article on Pat G. at Intel complaining about the industry needing to get away from CUDA and instead embrace open standards

Strange he didn't also say that the industry needed to get away from x86 and instead embrace open standards.

 

[Vanderlindemedia]

When are people other than Nvidia, OpenAI and a chosen few cloud providers supposed to start actually getting revenue out of AI?

It's all based on training large, huge if not infinite models at this point.

Once you have a solid product, you can rent it out through subscription(s), and use even that data for new training.

With direct chip cooling, voltages could be lowered, and thus making the energy requirement on that scale, in the hundred of thousands of watts less.

 

[Pierce2623]

It's all based on training large, huge if not infinite models at this point.

Once you have a solid product, you can rent it out through subscription(s), and use even that data for new training.

With direct chip cooling, voltages could be lowered, and thus making the energy requirement on that scale, in the hundred of thousands of watts less.

Yeah it’s been all about training large models since Hopper released a couple years ago. ChatGPT is still the only real monetizable project.

 

[hotaru251]

so proprietary cooling? meaning they'd have to upgrade EVERYTHING & redesign to accommodate.

When are people other than Nvidia, OpenAI and a chosen few cloud providers supposed to start actually getting revenue out of AI?

nvidia already has..
They sell the gpu's. They sell their cloud gaming (which uses their ai).

the rest of em are all racing to AGI as they know only one person wins the payoff.

 

[Vanderlindemedia]

Yeah it’s been all about training large models since Hopper released a couple years ago. ChatGPT is still the only real monetizable project.

I've been using both (ChatGPT + Grok) and i have to say Grok is far better then ChatGPT.

 

[wwenze1]

"These coolant distribution units (CDUs), such as the CoolIT CHx2000, are capable of managing up to 2 mW of cooling capacity,"

2MW not 2mW

 

[Crotus of Borg]

"...the company has invested in reference designs of sealed liquid cooling solutions to avoid the use of evaporative liquid cooling solutions, in an effort to preserve water"

Open-loop evaporative cooling is not used at the chip level or rack level. It is used to produce cold water at the building level which is used to cool the air or water supplied to the server racks. The chilling technology at the chip/ rack level and building level are basically independent.

https://datacenters.microsoft.com/w...ure_Modern-Datacenter-Cooling_Infographic.pdf

This is 100% correct and is the major fault of this whole article. The use of liquid cooling does not reduce water use at all unless the DC operator creates a primary fluid network independent of the racks that does not use water to cool the coolant at the coolant distribution unit. It can be done, absolutely, but you have to design for it.