News Nvidia's defeatured H20 GPUs in China sell surprisingly well — 50% increase every quarter in sanctions-compliant GPUs for Chinese AI customers

[Admin]

Despite major performance cut, Nvidia's H20 HGX brings the company billions in sales, according to a report

Nvidia's defeatured H20 GPUs in China sell surprisingly well — 50% increase every quarter in sanctions-compliant GPUs for Chinese AI customers : Read more

 

[Gu3sts]

Are these d"downgraded" at chip level, or at firmware level?

 

[jlake3]

Are these d"downgraded" at chip level, or at firmware level?

Specs are difficult to find, but it appears that H20 is a 78 SM configuration where the H100 comes in 114 SM and 132 SM versions. SMs have been fused off at the die level for many years and not unlockable via vBIOS flashing.

 

[bit_user]

One way to look at this news is that the sanctions are working to a significant degree. If it were a practical alternative for them to just buy H100 and H200's, I doubt Nvidia would be selling hardly any of these at all!

 

[Pemalite]

In the end, you just add more GPU's to get the same performance.
So the sanctions really aren't changing the landscape much.

 

[bit_user]

In the end, you just add more GPU's to get the same performance.
So the sanctions really aren't changing the landscape much.

Scaling isn't linear. There's a lot of communication involved in training AI models and the more compute nodes you need to exchange data with, the more overhead it adds.

 

[ottonis]

Of course it sells well. Just today, there was an article here on tomshardware.com about some guys running a LLM on a Windows 98 machine, powered by a single core Pentium II.
And with regards to training a model: it's much more important how the training data is selected and how the parameters are adjusted than the total time of processing. I don't care if the training lasts 58 days, 89 or 116 days - as long as it delivers great results.

 

[ottonis]

One way to look at this news is that the sanctions are working to a significant degree. If it were a practical alternative for them to just buy H100 and H200's, I doubt Nvidia would be selling hardly any of these at all!

Not sure about that. It all depends on the goal and aim of the sanctions. If the goal was to slightly slow down their efforts of developing large AI models, then yes, they were marginally successful. After all, with a slower/less capable compute unit, you can do everything that you can do with a more capable one, just slower. Heck, they could even pull a "Seti@Home"-like distributed computing approach and still develop everyting they want even without any nvidia card at all.

 

[bit_user]

Of course it sells well. Just today, there was an article here on tomshardware.com about some guys running a LLM on a Windows 98 machine, powered by a single core Pentium II.

That has nothing to do with anything. First, that was inferencing, not training. The main reason you need lots of powerful GPUs is for training.

Secondly, the quality of that model is too poor for literally anything, which made it just a transparently lame attempt to get some PR. It sort of worked, because they correctly predicted that a lot of people would just read the headline, but for the people who's attention actually could be relevant to their goals, they'll quickly lose interest upon seeing the details.

 

[bit_user]

Not sure about that.

Everyone seems to focus on the exceptions where sanctioned Chinese users got H100/H200 GPUs anyway. But, with a mass-production item, you're always going to have some slipping through the cracks, so that doesn't necessarily concern me. What concerns me is the overall volume, and these clearly wouldn't be selling if they could really get H100/H200 GPUs very easily.

Heck, they could even pull a "Seti@Home"-like distributed computing approach and still develop everyting they want even without any nvidia card at all.

No, you can't, for the reason I explained a few posts back. Training a large model requires way too much high-speed communication. An approach like that might take like a decade to train models like the one they did in 2 months.

 

[bit_user]

Specs are difficult to find, but it appears that H20 is a 78 SM configuration where the H100 comes in 114 SM and 132 SM versions. SMs have been fused off at the die level for many years and not unlockable via vBIOS flashing.

According to this, they did more than just disable SMs. They must've also significantly nerfed functional units within the SMs:

"When it comes to performance, HGX H20 offers 1 FP64 TFLOPS for HPC (vs. 34 TFLOPS on H100) and 148 FP16/BF16 TFLOPS (vs. 1,979 TFLOPS on H100)."

https://www.tomshardware.com/tech-i...s-new-ai-and-hpc-gpus-for-china-market-report
​

Depending on whether the specs were referencing sparse or dense tensors, the half-precision tensor performance got cut to just 7.5% or 15% of a H100. The capability they left notably intact was NVLink.