News Nvidia Hits Record $13.5 Billion Revenue on Explosive AI GPU Sales

[Admin]

Nvidia earns $10.3 billion selling high-performance compute GPUs for AI and HPC applications .

Nvidia Hits Record $13.5 Billion Revenue on Explosive AI GPU Sales : Read more

 

[JarredWaltonGPU]

I for one welcome our new AI overlords!

( It's a joke, just in case that's not clear:

View: https://youtu.be/8lcUHQYhPTE?si=y0H15oHZ5GR0_jqa&t=36
)

 

[gg83]

I for one welcome our new AI overlords!

( It's a joke, just in case that's not clear:

View: https://youtu.be/8lcUHQYhPTE?si=y0H15oHZ5GR0_jqa&t=36
)

Hey not for nothing, with the increasing complexity of human civilization AI might be the only way to keeps our "ship" afloat.

 

[Phaaze88]

/looks at other news threads
You don't say...
Gamers get bent, we(company and shareholders) can't hear you over the sound of all our ~money!!!
Ya helped build the company up, but it's so quick to get back-stabbed when the opportunity presents itself...

I for one welcome our new AI overlords!

( It's a joke, just in case that's not clear:

View: https://youtu.be/8lcUHQYhPTE?si=y0H15oHZ5GR0_jqa&t=36
)

No beep boops, or binaries?

 

[InvalidError]

The elephant in the room: how sustainable will that be?

 

[Phaaze88]

The elephant in the room: how sustainable will that be?

Doesn't matter. Made bank, will get away with it, and move on to the next big hype.

 

[Order 66]

Not sure how Nvidia's gaming revenue was up since hearing all the reports of 40 series gpus sitting on shelves. Nvidia must have sold a lot of 4090s.

 

[bourgeoisdude]

Not sure how Nvidia's gaming revenue was up since hearing all the reports of 40 series gpus sitting on shelves. Nvidia must have sold a lot of 4090s.

That and likely their margins are really high. Which is why I really hope AMD prices their new cards low enough to force NVIDIA to lower theirs. But I'm expecting that to not happen.

I really want to upgrade my 1070 ti and was close to pulling the trigger on the 6950 xt yesterday for $600 but I'm hoping for something that draws a bit less power. Also looking in the $500ish range. Hoping the 7800 xt will be there.

 

[bit_user]

The elephant in the room: how sustainable will that be?

It just depends on how well they can stay on top of the pile. AI isn't going anywhere, and there will always be demand for more training and inference performance.

Remember, they're making a concerted push for AI in cars and robots, as well - it's not only the datacenter they're after.

 

[qwertymac93]

It's 2024 already? 🤔

 

[bit_user]

It's 2024 already? 🤔

Companies' "fiscal year" often don't align with the calendar year. The fiscal year is just an accounting construct. I guess it's named after the calendar year that it overlaps with the most.

 

[TerryLaze]

Nvidia earns $10.3 billion

They didn't earn that amount, they sold product in that amount.
They earned $6,188 billion from all their markets combined.
ヽ(`Д´)ﾉ

 

[InvalidError]

Remember, they're making a concerted push for AI in cars and robots, as well - it's not only the datacenter they're after.

Sustainability isn't just about the growth of sales though. It also includes the ability to supply materials, the ability to secure power to run the whole thing, the ability for the environment to cope with all of that extra heat and other resource usage, etc.

Global warming and climate change say the environment may already have gone over the cliff. How many crypto-booms worth of AI stuff do you really want to add on top?

 

[JarredWaltonGPU]

Not sure how Nvidia's gaming revenue was up since hearing all the reports of 40 series gpus sitting on shelves. Nvidia must have sold a lot of 4090s.

https://cdn.mos.cms.futurecdn.net/MjrcinoSj9FzSMGq4EevYM.png

Look at the timeline. Massive (relative) profits on gaming during Q3'22 through Q1'23, then things dropped off a cliff for Q2'23 through Q4'23. That was roughly when RTX 40-series first launched, though getting production going probably took a bit (plus the lower tier GPUs weren't out). So, now that we're a year on from the RTX 4090 launch, supply and demand have ramped up some. But it's still ~30% lower than the peak in Q1'23.

 

[bit_user]

Global warming and climate change say the environment may already have gone over the cliff. How many crypto-booms worth of AI stuff do you really want to add on top?

It's not about what I want. It's about what I expect the market will demand. Personally, I hate how each new generation of GPUs (and CPUs) seems to burn ever more power, but that's where the competitive landscape and recent manufacturing tech are intersecting.

Anyway, it's fair to talk about energy scaling (both on the generation side and the heat dissipation aspect), because we're starting to face real problems, there. However, even if the selling point of new products is better perf/W more than absolute performance increases, that's potentially enough to keep sales brisk.

 

[TerryLaze]

Personally, I hate how each new generation of GPUs (and CPUs) seems to burn ever more power,

What are you even talking about!?
We had single core CPUs that had more than 100W difference between idle and full load and 226W for the whole system, now we have 230W for 16 cores from AMD (14w per core avg) and 253W for 8+16 cores from intel (10.5W per core avg). That's a huge trend DOWNWARDS in power usage.

Intel's Pentium 4 570J - Will 3.8GHz do the trick?

www.anandtech.com

 

[bit_user]

What are you even talking about!?

If you don't cherry-pick, but instead look at the trend lines, maybe you'll figure it out.

Source: https://github.com/karlrupp/microprocessor-trend-data
​

Of course, that's microprocessors, which was merely a parenthetical. The main point was about GPU power scaling, which is even worse. I have data on that, but not a plot. I'll update, when I do.

 

[TerryLaze]

If you don't cherry-pick, but instead look at the trend lines, maybe you'll figure it out.

So what I see there is the number of transistors increasing like crazy as well as the number of cores but the amount of power hasn't really changed since the early 2000 it increased a very small amount while cores and even single thread performance increased by a lot.

 

[InvalidError]

even single thread performance increased by a lot.

With a sudden slowdown from ~10X per decade for 1970-2006 down to about 5X over the 16 years from there to present... 60% longer for half the gain, a 3X slowdown. It may still be increasing but it is increasing much more slowly than it used to.

 

[TerryLaze]

With a sudden slowdown from ~10X per decade for 1970-2006 down to about 5X over the 16 years from there to present... 60% longer for half the gain, a 3X slowdown. It may still be increasing but it is increasing much more slowly than it used to.

Yes, but compared to how little the power increased, the single thread performance increase is a lot.

 

[bit_user]

the amount of power hasn't really changed since the early 2000 it increased a very small amount

As mentioned, there are scaling problems with increasing power too rapidly, but it's definitely creeping up.

The Y-axis is log-scale, meaning there's a factor of 10 between each pair of ticks. So, even what visually appears as a slight increase is actually pretty big. And there's definitely a positive slope - which should come as no surprise to any of us following the computer industry, over that time.

Finally, the data stops at 2021, meaning it won't include Sapphire Rapids or Genoa, both of which increased TDP relative to their predacessors.

Once I generate the GPU graph, I'll have a go at updating the data for this one (according to his same methodology for the other datapoints) and regenerating it.

 

[TerryLaze]

The Y-axis is log-scale, meaning there's a factor of 10 between each pair of ticks.

Yeah and there are black dots above the 10^2 mark which means the graph includes CPUs or systems with over 100 cores, like well over 100.
So you are bitter about CPUs with hundreds of cores using more that 100W or some imaginary number that you would like to be the max?!

 

[bit_user]

Yeah and there are black dots above the 10^2 mark which means the graph includes CPUs or systems with over 100 cores, like well over 100.

Yeah, a look at the underlying data shows some weird stuff after 2018. Before that, the outliers were both generations of Xeon Phi, which is debatable but fair enough to include. After that, he bizarrely included two data points for HPC GPUs (one AMD, one Nvidia), even though they can't possibly run SPEC int benchmarks!

He also included data points for Apple M1 and M1 Max, which is really strange, as they're laptop SoCs. Basically everything else included is a server CPU. They introduce outliers for no good reason.

So, I've removed those, added Milan, Genoa, Bergamo, Sapphire Rapids, and Grace. This is the result:

​

I couldn't exactly match the SPEC2017_int scores, used to compute the Single-Thread Perf, so there's an artificial dip in those scores. I also didn't try too hard to find transistor counts for the new samples. The only one I added was Milan, but I'm not even sure if that includes the IOD.

So you are bitter about CPUs with hundreds of cores using more that 100W or some imaginary number that you would like to be the max?!

I wouldn't say I'm bitter. I'd characterize it more as concern about efficiency, which I'd subdivide into two separate concerns. First, I'm concerned about systems being run too far above their efficiency sweet spot. For datacenter systems, I wouldn't say that's a major concern, since I know they're minding TCO and that should probably keep them in the more efficient range. My bigger concern is just that efficiency-scaling seems to be breaking down, overall.

Here is the power data plotted on its own. You can clearly see that it's still on the increase:

​

The two peaks are Genoa-X (400 W) and Nvidia Grace (144-core superchip), which I had to estimate because the 500 W total I found included all of the LPDDR5X memory.

 

[TerryLaze]

I wouldn't say I'm bitter. I'd characterize it more as concern about efficiency, which I'd subdivide into two separate concerns. First, I'm concerned about systems being run too far above their efficiency sweet spot. For datacenter systems, I wouldn't say that's a major concern, since I know they're minding TCO and that should probably keep them in the more efficient range. My bigger concern is just that efficiency-scaling seems to be breaking down, overall.

Here is the power data plotted on its own. You can clearly see that it's still on the increase:

TCO: If one 500W cpu can replace two 250W CPUs providing the same performance then you don't need to buy all of the infrastructure (PSU/MOB/ram etc) twice and you don't need to power all of the infrastructure twice.

Efficiency scaling isn't power in a vacuum increasing.
If multithread performance scales at least the same as power then everything is great because of above.
If power increases more and more for less and less performance THEN it's a problem.

 

[bit_user]

TCO: If one 500W cpu can replace two 250W CPUs providing the same performance then you don't need to buy all of the infrastructure (PSU/MOB/ram etc) twice and you don't need to power all of the infrastructure twice.

Except that you know as well as I do that if datacenters were only looking to buy machines on the basis of consolidating the same amount of processing power they already have, they'd probably be down to a single rack, by now. The problem is that compute demands go up at least as fast as density increases, which is why they build more & bigger datacenters and keep looking for better ways to cool them.

Efficiency scaling isn't power in a vacuum increasing.

I don't know exactly what you mean, by that.

If multithread performance scales at least the same as power then everything is great because of above.
If power increases more and more for less and less performance THEN it's a problem.

In general, I think the problem can be reduced to the breakdown in Dennard Scaling.

In the TSMC data I just posted 1 week ago, you can see this in the way that density is increasing faster than power reduction:

	16FF+ vs 20SOC	10FF vs 16FF+	7FF vs 16FF+	7FF vs 10FF	7FF+ vs 7FF	5FF vs 7FF
Power	60%​	40%​	60%​	<40%​	10%​	20%​
Performance	40%​	20%​	30%​	?​	same (?)​	15%​
Area Reduction	none​	>50%​	70%​	>37%​	~17%​	45%​

Source: https://www.anandtech.com/show/1417...chnology-pdk-drm-eda-tools-3rd-party-ip-ready​

That basically means you have to make smaller and smaller chips, as you move to each new node, if you don't want power consumption to go up. We actually saw this, back during Intel's "quad core era", which is the last time they achieved consistent TDP reductions from one generation to the next. As soon as they started adding more cores, the efficiency improvements were overwhelmed and TDPs started to go up.