News Nvidia criticizes AI PCs, says Microsoft's 45 TOPS requirement is only good enough for 'basic' AI tasks

[Admin]

Nvidia recently hosted an event highlighting the advantages of RTX GPUs in a number of applications, including content creation and gaming. It also took some time to disparage the idea of Intel and AMD promoting "AI PCs" that only offer 45 teraops of performance, saying that's only enough for basic AI workloads.

Nvidia criticizes AI PCs, says Microsoft's 45 TOPS requirement is only good enough for 'basic' AI tasks : Read more

 

[usertests]

The point is energy efficiency. There have been a lot of ARM chips out there with around 1-10 TOPS accelerators (such as the RK3566 or MediaTek Genio 1200). These can be useful, and now similar but even more powerful capabilities are coming to the x86/ Windows (Snapdragon) ecosystem.

If laptops or desktops are plugged in all the time, it's probably better to have a discrete GPU with much higher performance.

In the long run, INT4/INT8/BF16/etc. accelerators will become standard in all desktop PCs and consoles, with performance of 100 TOPS or higher, and they will get used instead of the GPU if the GPU's performance isn't needed. That could become relevant to gaming... in the 2030s.

 

[Notton]

I understand what nvidia is trying to do here.
Crypto went bust, so they want AI to replace that void in "gaming" GPU sales.
Thusly, they are pitching TOPS performance to the formerly-crypto-formerly-blockchain-now-AI crowd.

I hope their arrogance comes to bite them in the butt this time around.

 

[leoneo.x64]

Why don't they put a CPU / GPU toggle and let users decide? Battery life and power consumption hit less than the cost of acquiring new platform for NPU based CPUs. 45 TOPs certainly looks paltry when you add in the cost of acquiring new CPUs and therefore a new platform for some users. On the contrary, GPU based AI PC will make it truly flourish

and Intel should not talk about power efficiency after releasing power hogs for decades now. A frugal chip on a power guzzling monster CPU is marketing facade. Laughable Gelsinger strategy

 

[edzieba]

The point is energy efficiency. There have been a lot of ARM chips out there with around 1-10 TOPS accelerators (such as the RK3566 or MediaTek Genio 1200). These can be useful, and now similar but even more powerful capabilities are coming to the x86/ Windows (Snapdragon) ecosystem.

The question is, if accelerators of single/double-digit TOPS are actually of any value. Are there really any tasks of that scale that can be appreciably accelerated with such a minimal performance FFB? With 1000 TOPS of a dedicated GPU you can bring a task of hours or minutes down to seconds, which makes it an attractive user-oriented "press button, get result" feature. If your little on-die accelerator is just taking a task that takes hours and reducing it to a single hour, then that's just going to end up being a feature nobody uses.

I understand what nvidia is trying to do here.
Crypto went bust, so they want AI to replace that void in "gaming" GPU sales.
Thusly, they are pitching TOPS performance to the formerly-crypto-formerly-blockchain-now-AI crowd.

I hope their arrogance comes to bite them in the butt this time around.

"Tryng to do"? Nvidia have for the last several years been building said AI accelerator cards, revenue for which dwarfs gaming card (and the entire client card market) revenue and has for quite some time.
There isn't any "trying to do", this is what they have already done, through a decade of investment in CUDA, adding Tensor cores to all their cards, etc.

 

[hotaru251]

NPU vs Nvidia GPU performance vs energy consumption.

Thats the point of NPU.

Crypto went bust, so they want AI to replace that void in "gaming" GPU sales.

bad take.

Crypto busting doesnt harm them.
they sell more GPU for ai than they ever did for crypto.


i doubt gaming sales are even over 15% of their profit.

When you have companies like tesla buying massive amounts of ur ai focused gpu's that cost like 10 grand a pop....

Jensen already said Nvidia is not a gpu company anymore its an ai company they just havent changed name.

Want to do ai stuff? Nvidia is basically ur best option atm (and they know it hence their attitude) if you dont care about anything but end result.

now if u aren't that type of customer (which most arent)? NPU are cheaper & more effective at a lower energy cost.
Most people who use "ai pcs" arent needing the power from a 4090+ tier of gpu.
They are fine w/ a small npu.


Nvidias benefit has always been others made their stuff work with nvidia features. (mainly cuda)
Some are now trying to stop that and make stuff more open source but if it works long term nobody knows.

and if NPU's show as being the future over ur gpu doing it? You can bet nvidia will be there as they have a bunch of really smart people who can make it.

 

[Amdlova]

Only thing I want IA do is help with my torrents

 

[usertests]

Only thing I want IA do is help with my torrents

Like what? Maybe future versions of the Tribler torrent client?

Researchers Showcase Decentralized AI-Powered Torrent Search Engine * TorrentFreak

Researchers from Delft University of Technology plan to amplify their BitTorrent client "Tribler" with decentralized AI-powered search.

torrentfreak.com

The question is, if accelerators of single/double-digit TOPS are actually of any value. Are there really any tasks of that scale that can be appreciably accelerated with such a minimal performance FFB? With 1000 TOPS of a dedicated GPU you can bring a task of hours or minutes down to seconds, which makes it an attractive user-oriented "press button, get result" feature. If your little on-die accelerator is just taking a task that takes hours and reducing it to a single hour, then that's just going to end up being a feature nobody uses.

Probably some lighter use cases like image recognition/real-time filtering, small LLMs (Copilot), Stable Diffusion, etc. are fine with the upcoming baseline of ~40-50 TOPS.

QNAP TS-133 1-bay NAS leverages Rockchip RK3566 AI capabilities for object and face recognition

We’ve seen several hardware devices based on Rockchip RK3566 AIoT SoC that do not make use of the key features of the processor. But QNAP TS-133 1-bay NAS is different, since it relies on the native SATA and Gigabit Ethernet interfaces for network storage, and the built-in NPU is leveraged to accelerate object and face recognition by up to 6 times.

That's using the ~0.8 TOPS NPU in the RK3566. "Up to 6 times" the performance is probably in relation to either the relatively slow quad-core Cortex-A55, or the Mali-G52 iGPU.

I'm not too keen on the 9-16 TOPS NPUs found in Meteor Lake, Phoenix, and Hawk Point, since they can be outperformed by the accompanying iGPU. That should change this year once we see around 50 TOPS XDNA2 in Strix Point. Future versions of these NPUs are likely to hit at least low triple digit TOPS.

 

[Neilbob]

If your little on-die accelerator is just taking a task that takes hours and reducing it to a single hour, then that's just going to end up being a feature nobody uses.

I know I sound completely ignorant here, and I am well aware I'm almost blindly cynical and skeptical about the Ayy-Eye jiggery-pokery, but please tell me exactly what task/s.

What are the tasks that will be so ubiquitous that such a large number of consumers (I can't stress that word enough) will require Nvidia's rather pricey version of TOPS (whatever the heck they are)? What might I need to do that could take hours?

If I need to 'generate' some kind of image, I can imagine I'd only do it a couple of times out of curiosity and then never bother again. I'm just not seeing what all the fuss is meant to be about for someone like me in everyday-world. Please educate me.

 

[ivan_vy]

some of us don't need 4090 IA power, we need better videocalling, noise filter for audio and video, light photoediting like color correction and background removal, we live in MS Office world, for everything else we can use the cloud services, power users, researchers and cutting edge content creators they will eventually go to Nvidia, that's until open source options make a better cost benefit ratio.
for gaming would be nice to have AI lighting (RT or PT are too HW demanding) and buttery smooth framerates...it looks like where P5Pro is aiming to go.

 

[Eximo]

I think the key is that software developers will be encouraged to improve battery life and use the appropriate silicon to maximize that. Generally it would be seamless to the user.

I could also see local processing for things like predictive text, search, or action prediction. For instance, loading up the appropriate code into memory anticipating the user's next actions. Makes the user experience better, and reports up to the big AI on your use habits of course.

 

[ivan_vy]

Makes the user experience better, and reports up to the big AI on your use habits of course.

big companies gotta make you pay subscription for better AI companions and get those sweet sweet data.

 

[Giroro]

Nvidia could have a hundred billion trillion TOPS. Nobody knows what a TOPS is, and nobody cares.
The only AI question anybody cares about is "Can I easily run Chat GPT and image generation locally, uncensored, without a subscription, forever?" Nvidia's answer is still no.
Even then, that's not a question that many people are asking. But until that happens, the hardware doesn't matter at all to the end user.

 

[mtrantalainen]

The point is energy efficiency. ...
In the long run, INT4/INT8/BF16/etc. accelerators will become standard in all desktop PCs and consoles, with performance of 100 TOPS or higher, and they will get used instead of the GPU if the GPU's performance isn't needed.

I predict that we're going to see 50-100 TOPS of AI performance out of CPU and in addition to that, you need proper GPU for graphics.

The AI in CPU is going to calculate AI stuff for games while GPU is working on graphics.

100 TOPS is still 100 trillion operations per second and that's quite a lot of computation even though each computation most be one of very limited operations.

And let's not forget that the Microsoft given baseline range is baseline. That's no different from H.264 baseline requiring system to have at least 24 megabytes of RAM available for video processing. Modern Windows has baseline requirement of 4000 megabytes of RAM for the operating system alone!