News Keller and Koduri headline the Beyond CUDA Summit today — AI leaders rally to challenge Nvidia's dominance

[Admin]

The Beyond CUDA Summit commences tomorrow, bringing together key leaders and engineers in the industry to explore the future of AI with alternative platforms.

Keller and Koduri headline the Beyond CUDA Summit today — AI leaders rally to challenge Nvidia's dominance : Read more

 

[ezst036]

We have 5 standards, each competing with the other and no clear winner. Here's an idea, what if everybody could rally behind just one standard?

*creates new standard*

Afterwards:

There are 6 competing standards.

 

[bit_user]

ROCm only supports a small selection of modern GPUs while CUDA maintains compatibility with hardware dating back to 2006.

This needs two big footnotes.

1. AMD might finally be getting to grips with the technical problems that have limited the ROCm support window. We'll see.
2. While CUDA works on older hardware, the latest CUDA versions do not. IIRC, each CUDA version has a minimum driver requirement and once a Nvidia GPU moves into legacy support status, it stops getting newer drivers. They still provide some bugfixes to the legacy support drivers, but you won't be able to run newer CUDA versions on it.

Also, I think CUDA is a little bit of a trap that AMD (and, to some extent, Intel) walked right into. You can't build a better CUDA. The best CUDA-like thing will always be Nvidia's CUDA.

There was a brief window of time where it looked like OpenCL might be able to gather enough momentum, but that fell apart as Apple and Google withdrew support for it, and then AMD got distracted by HSA and then HIP. At the time, Intel wasn't really a GPU player. More recently, we have Rusticle, which extends OpenCL to most GPUs supported by the open source Mesa userspace graphics API framework. While Rusticle offers the tantalizing promise of near-ubiquitous out-of-the-box OpenCL support, it comes very late in the game.

Finally, CUDA is not the ideal API for AI. The CUDA programming model is suited for SIMD + SMT, which is great for fairly general-purpose parallel programming, but too wasteful for AI. I expect Nvidia to move beyond CUDA, for their future training chips! Maybe Rubin is that thing, maybe not.

 

[Giroro]

Koduri is directly responsible for putting both AMD and Intel into terrible positions to compete against Nvidia. Not just in AI, but in everything.
Why would anybody expect him to have a strategy to fight Nvidia? He's the one who cleared a path for Nvidia to build a monopoly so easily. He's incompetent at best, maybe even a saboteur.
I don't know if he's secretly on Nvidia's side, but he definitely has the poison touch - which real leaders at this summit would be wise to avoid.

 

[bit_user]

Koduri is directly responsible for putting both AMD and Intel into terrible positions to compete against Nvidia. Not just in AI, but in everything.

The poor showing of Vega and Alchemist both happened under him, which does look very bad. Not to mention the excessive delays and horrible yields of Ponte Vecchio (i.e. the Intel Datacenter Max GPUs). Plus, Intel canceled a mid-range datacenter GPU, before it reached the market - what were they thinking, even starting with 3 simultaneous architectures, in their plan - 4, if you count the iGPUs??

So, yeah. I tend to think he had a lot to do with bad decisions behind both of those product lines. However, to be fair, we should recognize that RDNA got started under Koduri, as well. So, I might still listen to what he has to say, but also take it with a grain of salt.

Why would anybody expect him to have a strategy to fight Nvidia? He's the one who cleared a path for Nvidia to build a monopoly so easily.

I wonder who actually signed off on AMD's software strategy. Because I think ROCm's rough track record had a lot more to do with why AMD hasn't been able to effectively serve the AI training market than AMD's hardware. MI300X is actually good hardware, but ROCm is what's been holding it back.

He's incompetent at best, maybe even a saboteur.
I don't know if he's secretly on Nvidia's side, but he definitely has the poison touch - which real leaders at this summit would be wise to avoid.

I think you're really stepping over the line. Nvidia is a ruthless competitor and this is a tough business. To beat them, you need to be exceptional. I think Koduri just wasn't quite up to the challenge.

Specifically, I think one of his main failings was that he was too ambitious - he didn't properly account for the limitations of his teams, so he over-promised and under-delivered. The smarter thing would've been to try to do more with less. At Intel, he should've never agreed to try and leap-frog Nvidia in a single generation. If you go back and look at the eye-watering complexity of Ponte Vecchio, it looks like it was designed by a mad man!

 

[blppt]

Any time Jim Keller is involved in anything, the tech world takes notice. Forget Koduri.

 

[hotaru251]

its really simple...just don't use/support cuda in future stuff. make a standard agreed by the ones involved that bars using cuda same way nvidia bans others using cuda.

 

[ekio]

Vulkan is an amazing API, available on all machines easily.
I would try to improve Vulkan Compute instead of creating a new standard again nobody will use.

 

[bit_user]

its really simple...just don't use/support cuda in future stuff. make a standard agreed by the ones involved that bars using cuda same way nvidia bans others using cuda.

Which standard? Intel has been pushing oneAPI; AMD has been pushing HIP.

If the problem we're really trying to solve is AI, I've argued that you don't actually want something CUDA-like, at all. Then again, there are already AI standards like ONNX, though I don't know if they have any support for custom layers (that's where CUDA gets its nose under the tent).

 

[bit_user]

Vulkan is an amazing API, available on all machines easily.

It is not available on many purpose-built AI chips or FPGAs (which do offer support for OpenCL). The machines that support it are only GPUs. On a CPU, you can run Lavapipe, but I'm not even sure if that scales very well.

I would try to improve Vulkan Compute instead of creating a new standard again nobody will use.

Vulkan is an absolute mess of different versions and extensions though. It's also a major pain to use.

It doesn't even have a standard shading language, which is a big part of the problem CUDA solved, although Microsoft has recently made HLSL free to use with Vulkan.

Next, I think SPIR-V is too GPU-centric. It doesn't address the concerns I have about CUDA-type solutions being a poor fit for AI.

Finally, Vulkan is not a good solution for scientific or financial processing, due to significantly lower precision guarantees than OpenCL provides.

If the way forward is through Vulkan, it will still need a lot of work. That much, I can say for sure.

 

[wwenze1]

You know you've won when you have people getting together to talk about "beyond you" and never invited you.

Also isn't CUDA a trademarked term? Why are they using this term for this event?

 

[bit_user]

Also isn't CUDA a trademarked term? Why are they using this term for this event?

As you say, it just flatters Nvidia that they're talking about CUDA. Nvidia is probably smart enough not to score an own-goal by trying to sue them for trademark infringement. That would just make them look petty and insecure.

 

[jrhansen]

The problem is to create an api that can be used on all kind of gpu or usable chip silicon for enhancing mathematical calculations, and then get it out to the people who designs software. It's hard and takes a long time.
You can watch how far they have come to make an alternative to Cuda for making VST plugins for specific audio manipulations. We have known right from the start of the first DAW software and the first ideas of Cuda that this would be excellent for many audio things that in the 80's and 90's was done with expensive DSP processors, because a standard x86 processor was not powerful enough to run more than a few audio tracks in just CD audio format.
Unfortunately CUDA don't seem to be very effective in this area and is again Nvidia only. So some bright heads have come up with their own api that runs on both AMD, Nvidia, Apple Silicon and Intel, but it takes time to make it commonly available and out to all the developers. Most seem to be running the safest way and just using the normal VST api that runs on the cpu, when at least some plugins would highly benefit on a powerful GPU that often sits doing nothing but showing a standard windows or Mac Desktop on an otherwise well equipped Workstation in CPU cores, Ram and SSD for doing serious audio recording and manipulations.

And the same goes for AI. We need a standard that embraces all available hardware standards in a common api so everyone can be included, no matter if your just dabbling in AI at home with a standard home device or a serious server park system, so that if you get the right idea can easily move it to a more "pro" setup.

 

[bit_user]

The problem is to create an api that can be used on all kind of gpu or usable chip silicon for enhancing mathematical calculations, and then get it out to the people who designs software. It's hard and takes a long time.

OpenCL already is that. It's more than 15 years old. To make a success of such a project requires more than just creating the standard. You need a big industry player, like Google or Microsoft, who will force hardware makers to support it.

And the same goes for AI. We need a standard that embraces all available hardware standards in a common api so everyone can be included, no matter if your just dabbling in AI at home with a standard home device or a serious server park system, so that if you get the right idea can easily move it to a more "pro" setup.

Well, I mentioned ONNX:

https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange
​

And there's also NNEF.

https://en.wikipedia.org/wiki/Neural_Network_Exchange_Format
​

I know these are supported to some degree, but I lack the expertise to say where/why they're falling short. I have a hunch that perhaps they don't address the need for custom layers or certain network types.

​

 

[EzzyB]

its really simple...just don't use/support cuda in future stuff. make a standard agreed by the ones involved that bars using cuda same way nvidia bans others using cuda.

You're really ignoring business realities here. AI is a cutthroat market and CUDA is king. The winner in AI gets to set the narrative. None of this has anything to do with AI assistants or breakthroughs. Absolutely none of the competitors are willing to wait, and just saying "don't use CUDA" is going to fall on deaf ears.

Musk wants a "non-woke" AI
Alibaba wants an AI who's answers are "properly socialist"

Everyone has an agenda.

That's what's really going on here, that's why it's so important. If someone emerges from this huge AI spending orgy with a dominant AI platform, they get to set the narrative for AI in the future. The idea of some kind of benevolent, utopian AI is ludicrous. That AI's views are going to reflect whatever the hell it's creator, who spent so much money, wants it to and every player is terrified it won't be them.