News AMD-Friendly AI LLM Developer Jokes About Nvidia GPU Shortages

[Admin]

AMD LLM accelerator partner Lamini poked fun at Nvidia GPU solutions for their lengthy order lead times, noting that AMD GPUs are on the grill and ready for use, and comparing ROCm and CUDA capabilities.

AMD-Friendly AI LLM Developer Jokes About Nvidia GPU Shortages : Read more

 

[scottslayer]

Ah yes, a social media post by someone who has AMD as their customer.
Thanks for the quality "news" yet again Tom's.

 

[The Hardcard]

Ah yes, a social media post by someone who has AMD as their customer.
Thanks for the quality "news" yet again Tom's.

Actually, statements from companies, verifying the functionality of ROCm software stack is important news. The biggest part of the AI battle is software.

A lot of companies have matrix math (tensor) units that are capable of doing everything iNvidia GPUs can do. (minus sparsity accelerators added to Ampere and the transformer hardware added to Hopper) . The software stack and documentation are what is lacking for a number of companies, including AMD.

The post makes it more likely the AMD will get enough of this together long before Nvidia can satisfy demand for its hardware. This means that one: AMD is going to get to jam fists into the AI loot box and two: consumers will have more access given that AMD already has an established consumer side of this business.

 

[Makaveli]

Actually, statements from companies, verifying the functionality of ROCm software stack is important news. The biggest part of the AI battle is software.

A lot of companies have matrix math (tensor) units that are capable of doing everything iNvidia GPUs can do. (minus sparsity accelerators added to Ampere and the transformer hardware added to Hopper) . The software stack and documentation are what is lacking for a number of companies, including AMD.

The post makes it more likely the AMD will get enough of this together long before Nvidia can satisfy demand for its hardware. This means that one: AMD is going to get to jam fists into the AI loot box and two: consumers will have more access given that AMD already has an established consumer side of this business.

The post yours above I would classify as " Can't see the forest for the trees"

And your follow up post was excellent!

 

[scottslayer]

Apologies to all organic posters, I did not scroll down to see the actual news that is in the last half or so of the article because I was severely unimpressed with the whatnot in the first half.
Consider me appropriately chastised.

 

[ivan_vy]

Actually, statements from companies, verifying the functionality of ROCm software stack is important news. The biggest part of the AI battle is software.

A lot of companies have matrix math (tensor) units that are capable of doing everything iNvidia GPUs can do. (minus sparsity accelerators added to Ampere and the transformer hardware added to Hopper) . The software stack and documentation are what is lacking for a number of companies, including AMD.

The post makes it more likely the AMD will get enough of this together long before Nvidia can satisfy demand for its hardware. This means that one: AMD is going to get to jam fists into the AI loot box and two: consumers will have more access given that AMD already has an established consumer side of this business.

also means Nvidia hardware is no longer essential, AMD will fill the needs of many consumers and that leads to lower prices on hardware, a win for all, except for some leather jacket enthusiasts.

 

[Steve Nord_]

Would have liked to see more links to those cards/frames on sale, oh well I will just build out the solar panels and PowerWall -ish storage until my magic grill pans out.

 

[Li Ken-un]

Here’s to the end of Nvidia’s grip on the market.

Normally, hardware manufacturers use software to add value (and thus sell more) hardware. Nvidia, sells the hardware and then charges subscriptions on top of that for already baked in hardware functionality. LOL!

 

[russell_john]

The problem here is Nvidia isn't using CUDA for LLMs, it's using tensor core neural networks for LLMs and AMD basically has nothing that can compete in that area.

Nvidia having a shortage of GPUs is actually a good thing, it means they are selling them faster than they can make them ...... AMD having plenty on hand means poorer sales and overstock ..... Same overstock we have seen in the consumer markets that caused a 8 month delay in the release of the 7800XT .... Every discounted 6950XT sold this year equals one less 7800XT that will be sold this year.

Nice try at spin, I'm sure it fools a lot of people who don't really understand how market dynamics work. AMDs overstock of last gen products has made them even less competitive because they were forced to cannibalize the sales of their current gen.

 

[Blacksad999]

Oh, nice!!

These companies can now buy FIVE mediocre GPU's to do the work of one!! What a score!! lol I'm sure they're just as "power efficient" as the rest of the AMD lineup. XD

 

[The Hardcard]

The problem here is Nvidia isn't using CUDA for LLMs, it's using tensor core neural networks for LLMs and AMD basically has nothing that can compete in that area.

Nvidia having a shortage of GPUs is actually a good thing, it means they are selling them faster than they can make them ...... AMD having plenty on hand means poorer sales and overstock ..... Same overstock we have seen in the consumer markets that caused a 8 month delay in the release of the 7800XT .... Every discounted 6950XT sold this year equals one less 7800XT that will be sold this year.

Nice try at spin, I'm sure it fools a lot of people who don't really understand how market dynamics work. AMDs overstock of last gen products has made them even less competitive because they were forced to cannibalize the sales of their current gen.

Tensor cores are hardware matrix math units. CUDA is a software stack to do math on that hardware. CUDA is used for machine learning tasks, including LLM‘s. While tensor cores can be used without CUDA, no one has invested the tens of thousands of engineering man hours it would take to write an alternative software stack to drive them. For now, CUDA is the only way the tensor cores are used.

AMD has had matrix math units in their data center GPUs for nearly three years, and RDNA 3 also has them.

You also don’t need tensor/matrix units to run neural networks. They run on anything that can do math - GPU shaders, and even in CPUs. They just require significant amounts of data movement without matrix units, using up critical memory bandwidth that the main bottleneck in most neural network performance - LLMs in particular.

 

[gambeg]

Oh, nice!!

These companies can now buy FIVE mediocre GPU's to do the work of one!! What a score!! lol I'm sure they're just as "power efficient" as the rest of the AMD lineup. XD

Clearly, you don't know about LLM inference and training. (I even made a new account just to retort this lol)

I bet you don't read the whole article, the bottleneck in LLM is VRAM, that's why a 2016 P40 is still popular within LLM community. This thing might be slow for today's standard, but it got 24gb VRAM.

As for your lame attemp at sarcasm, the article clearly noted the Instincts got 50% more VRAM, so you are comparing a 80GB H100 against 5x128GB MI250X? a H100 won't even fit a 70B full-precision model.

It's also worth noting that Nvidia barely increased the VRAM against previous generation, they are clearly getting comfortable, just wait until they got Intel treatment, and I'm all for it!

 

[ivan_vy]

Oh, nice!!

These companies can now buy FIVE mediocre GPU's to do the work of one!! What a score!! lol I'm sure they're just as "power efficient" as the rest of the AMD lineup. XD

is not only about sales, think of the AMD cards already in the hands of companies that want to step into AI bandwagon, don't need to buy NVIDIA accelerators anymore to try it, also some companies are sitting in lots of AMD hardware and can offer the service like Google Stadia.

 

[gg83]

this is pretty awesome.

 

[edzieba]

Clearly, you don't know about LLM inference and training. (I even made a new account just to retort this lol)

I bet you don't read the whole article, the bottleneck in LLM is VRAM, that's why a 2016 P40 is still popular within LLM community. This thing might be slow for today's standard, but it got 24gb VRAM.

As for your lame attemp at sarcasm, the article clearly noted the Instincts got 50% more VRAM, so you are comparing a 80GB H100 against 5x128GB MI250X? a H100 won't even fit a 70B full-precision model.

It's also worth noting that Nvidia barely increased the VRAM against previous generation, they are clearly getting comfortable, just wait until they got Intel treatment, and I'm all for it!

Total memory available is important for inferencing, but for training the computational workload (i.e. TOPS) is more important. Companies buying up vast numbers of H100s are buying them for training, not for inferencing. For inferencing, half the time that's not even offloaded to GPUs, but to dedicated NNPs instead.
Nvidia are not increasing RAM pools because that's not what their customers are demanding. And their customers are demanding so hard that supply cannot keep up, so Nvidia appear to have targeted their workloads rather well.

 

[derekullo]

The Miata can be competitive and haul just as much as a Ford F250, particularly if you have enough of them.