News Nvidia Makes 1,000% Profit on H100 GPUs: Report

[JamesJones44]

The price in this circumstance is because there's literally no other choice at that level which means nobody is "choosing" this so much as hedging their bets that it will pay off. The only choice they're actually making is that not buying it would be worse.

There are plenty of choices. If you don't want/need to train/infer at the absolute best possible speed or game at the absolute best possible frame rate there are viable alternatives. Nvidia is simple a generation ahead of the competition at the high end. There is no reason to punish them for their competitors lack of vision or staffing. History has shown that regulation in highly fluent areas like hardware technology are often self defeating and in most cases by the time they are implemented irrelevant. All you do is waste time, money and energy on something that will change in 12 months anyway.

This isn't cable internet where there are built in monopolies with no chance for competition. Trust me, with the money Nvidia is making right now there are several companies working on catching up. Several have already started to design their own silicon (hello Google, Amazon, Qualcomm, etc.) for AI training. For gaming GPUs you are talking about a niche market, 99.9% of the people out there could care less if they had a 4090 or integrated graphics from the Kaby Lake era. If you are going to regulate niche markets then you might as well complain about Gucci, Ferrari, the four seasons hotel, etc.

 

[JamesJones44]

1000% is 10x.
Elementary school mathematics.
100% is one. 100%+100%+100%+100%+100%+100%+100%+100%+100%+100% is ten.
That'll be 15 dollars.

You caught my typo! It's supposed to be 1000%. If it were 1000x and people were actually buying them hand over fist Jensen Huang would probably dance in the street naked with joy.

 

[Peter Ferrari]

For computer GPUs you are talking about a niche market, 99.9% of the people out there could care less if they had a 4090 or integrated graphics from the Kaby Lake era.

5 billion people in the world have internet connection. steam has 120 million users. Is what you're saying true or not ?
And do all people with an igpu really don't care about having a better card or just have other priorities/can't afford one ?

Jesus Christ, you and hyperboles........

 

[Jaxstarke9977]

To prevent a company from ripping off their customers.

Do you think 1,000% Profit on H100 is fair to the consumer / customer?

No one is forcing anyone to buy one. If you don't like a company making a profit off of what they sell then shop elsewhere.

 

[Peter Ferrari]

You caught my typo! It's supposed to be 1000%. If it were 1000x and people were actually buying them hand over fist Jensen Huang would probably dance in the street naked with joy.

so what do you think they're making ? 10x or more like 5x ?

 

[JamesJones44]

5 billion people in the world have internet connection. steam has 120 million users. Is what you're saying true or not ?
And do all people with an igpu really don't care about having a better card or just have other priorities ?

Jesus Christ, you and hyperboles........

IDK, how many of those 120 million care about using the highest settings on a 4K monitor with ray tracing enabled?

I would love to see stats on the subject.

 

[Peter Ferrari]

IDK, how many of those 120 million care about using the highest settings on a 4K monitor with ray tracing enabled?

I would love to see stats on the subject.

I think 10% of that is enough for you to turn your business of selling gpus into a gold mine, if you make as much profit on one card sold as nvidia does. And nvidia have like 80% of steam gpus or more.
And going back to practising percentages, how much do you think they're making on each h100 ? You said 100x earlier, but that was a typo, so 100% (2x) right ? If it costs them 3300 to make one, they're not selling for 25-30 thousand, but no more than 7.000, yes ? Imo it's more than that.

 

[JamesJones44]

so what do you think they're making ? 10x or more like 5x ?

Just based on their overall gross margin, I would bet 2 or 3x. That's a moving target though of course because the more volume they push the more the non recurring costs get amortized off. If they run it for 2 or 3 more years it might end up being 5x or 6x if volume stays high enough.

 

[Peter Ferrari]

Just based on their overall gross margin, I would bet 2 or 3x. That's a moving target though of course because the more volume they push the more the non recurring costs get amortized off. If they run it for 2 or 3 more years it might end up being 5x or 6x if volume stays high enough.

I doubt it's only 2-3x. But we'll never know. Like you said, r/d is part of it, so is software. But once you develop it and it pays off, it's like free to profit off it as long as you ca make it and sell it. the 10x number is just hardware price. If r/d for hopper paid off with selling ampere (and it more than did with all the mining sales alone, not to mention a100 dgx), it might as well be correct to assume they make 10x off every h100.

 

[JamesJones44]

I think 10% of that is enough for you to turn your business of selling gpus into a gold mine, if you make as much profit on one card sold as nvidia does.
And going back to practising percentages, how much do you think they're making on each h100 ? You said 100x earlier, but that was a typo, so 100% (2x) right ? If it costs them 3300 to make one, they're not sellinmg for 25-300 thousand, but no more than 7.000, yes ? Imo it's more than that.

I said closer to 100% than 1000% percent in my original post. Not meaning to imply 100% at the number, only that it's likely less than 500%. As I said in my other post, just based on their financial statements I would guess in the 2 to 3x range. This is also factoring in R&D, support, etc. On a pure component based cost, I wouldn't be surprised if it is 10x. Apple for example usually only has a bill of materials that is about 10%-15% of the sales price, but they usually only book 43% of that after all costs are factored in (meaning of $1000 phone they book about $430 in profit, even thought the BoM is $150 for example).

 

[Peter Ferrari]

I said closer to 100% than 1000% percent in my original post. Not meaning to imply 100% at the number, only that it's likely less than 500%. As I said in my other post, just based on their financial statements I would guess in the 2 to 3x range. This is also factoring in R&D, support, etc. On a pure component based cost, I wouldn't be surprised if it is 10x. Apple for example usually only has a bill of materials that is about 10%-15% of the sales price, but they usually only book 43% of that after all costs are factored in (meaning of $1000 phone they book about $430 in profit, even thought the BoM is $150 for example).

Don't financial statements also need to include things like paying workers and paying for building maintanance, running a business in general ? Doesn't mean their cards don't sell for 10x of what they cost to make, just means they need to invest a lot to make a lot. To know what the profit is, you'd need to know the amount of items sold.
Just don't know how you can know this from financial statements.

 

[thestryker]

There are plenty of choices. If you don't want/need to train/infer at the absolute best possible speed or game at the absolute best possible frame rate there are viable alternatives. Nvidia is simple a generation ahead of the competition at the high end. There is no reason to punish them for their competitors lack of vision or staffing. History has shown that regulation in highly fluent areas like hardware technology are often self defeating and in most cases by the time they are implemented irrelevant. All you do is waste time, money and energy on something that will change in 12 months anyway.

This isn't cable internet where there are built in monopolies with no chance for competition. Trust me, with the money Nvidia is making right now there are several companies working on catching up. Several have already started to design their own silicon (hello Google, Amazon, Qualcomm, etc.) for AI training. For gaming GPUs you are talking about a niche market, 99.9% of the people out there could care less if they had a 4090 or integrated graphics from the Kaby Lake era. If you are going to regulate niche markets then you might as well complain about Gucci, Ferrari, the four seasons hotel, etc.

You managed to go on a two paragraph rant that has nothing to do with anything I said, good job.

 

[JamesJones44]

Don't financial statements also need to include things like paying workers and paying for building maintanance, running a business in general ? Doesn't mean their cards don't sell for 10x of what they cost to make, just means they need to invest a lot to make a lot. To know what the profit is, you'd need to know the amount of items sold.
Just don't know how you can know this from financial statements.

Gross margin/profit is revenue - cost of goods sold. It's a marker for how much they are making on their products before taking out taxes, office costs, R&D, etc. but it does account for packaging, shipping, etc. It does tell you how much it costs for them to make the products they sell. It's not a direct link by any means, especially when there are things in there like software which traditionally is very high margin. However, it gives an indication and by comparing that number and number quoted in this article for example with previous numbers Nivida has reported gives some indication. If Nivida was really earning 10x per H100 or 27K their gross margin would be almost 90% (30k - 3k -> 27k / 30k) for those cards, it would also likely be reflected in their overall numbers. However, their gross margin last quarter wasn't even their highest ever and has been lower than anytime in the past 10 years. Now maybe all of their other products are just doing horribly, IDK, but just looking at that makes me think their earning closer to what they always have for their products instead of earning 10x or 8x on hardware. Most hardware companies are in the 2 to 5x range for gross margins (some companies do break this out. Apple used to be really good about breaking out hardware margins. They don't do it as much these days now that hardware sales have slowed).

NVIDIA Profit Margin 2010-2024 | NVDA

Current and historical gross margin, operating margin and net profit margin for NVIDIA (NVDA) over the last 10 years. Profit margin can be defined as the percentage of revenue that a company retains as income after the deduction of expenses. NVIDIA net profit margin as of July 31, 2024 is...

www.macrotrends.net

Computer Hardware Industry Profitability, Gross Margin, Net Margin, Cash Flow Margin, ROE

Gross Margin, Net Margin, Cash flow Margin and Roe of companies within Computer Hardware Industry - CSIMarket

csimarket.com

 

[vehekos]

This is a shockingly ignorant post that doesn't seem to grasp how the real world works.

You are the only "shockingly ignorant" here, and do not understand how the world works.

The price in this circumstance is because there's literally no other choice at that level which means nobody is "choosing" this so much as hedging their bets that it will pay off. The only choice they're actually making is that not buying it would be worse.

That's irrelevant. The important thing is to discover the value of things. If somebody is willing to pay 10,000 for something that used to cost 100, that's a great discovery: the card is far more valuable than it was thought, hence when nvidia makes a lot more money from selling it, it can now buy more resources, with that profit, like more wafers, more time, better experts. Those have to be taken from alternative production, like gameboy consoles.

Because it has been discovered that nvidia cards are more valuable, society can spend limited resources making the most valuable stuff.

If all of humanity could make products that are worth 1000% than his cost, humanity would get super rich in no time.

Making a profit in the free market is being a philantrophe.

 

[bit_user]

wonder how long before nvidia starts building their own fab for their datacenter/ai lineup

Never. Fabs are more expensive than ever and require an extreme amount of expertise and extremely long ramp-up times.

Intel has been moving the other direction and now runs their fabs as a separate business, because it's so expensive they have trouble supporting their fabs even on the back of their revenue.

Basically, only governments have enough money to get into the semiconductor fabrication game.

 

[DavidLejdar]

Somewhat curious, if it isn't the crypto of the business world. Not saying that there isn't any potential to AI. But e.g. the company, I work for, pronounced in the news, that the company will have a company AI by the end of this year (which will apparently be based on the "Luminous" language model by Aleph Alpha - which supposedly does a better job at referencing the sources). And one of the first things it will do is to aid employees with search in a company's database. Meanwhile, a company-internal hotline, with actual humans providing such aid, could easily compete in costs with what this AI is supposed to be capable of. And in this context I wonder, whether someone in management skipped a cost-benefit analysis.

 

[Peter Ferrari]

Never. Fabs are more expensive than ever and require an extreme amount of expertise and extremely long ramp-up times.

Intel has been moving the other direction and now runs their fabs as a separate business, because it's so expensive they have trouble supporting their fabs even on the back of their revenue.

Basically, only governments have enough money to get into the semiconductor fabrication game.

I think it must have crossed their mind.

If Nivida was really earning 10x per H100 or 27K their gross margin would be almost 90% (30k - 3k -> 27k / 30k) for those cards, it would also likely be reflected in their overall numbers.

nvidia sell more than just h100. I don't think overall gross margin excludes the possiblility of them selling this piece of ai hotcake at 10x the production cost.

Meanwhile, a company-internal hotline, with actual humans providing such aid, could easily compete in costs with what this AI is supposed to be capable of. And in this context I wonder, whether someone in management skipped a cost-benefit analysis.

Do they work full-time at the office, with all the benefits, or fee for task from home ?
Ppl need parking space, office space, lunch area, someone to clean the office/toilet/kitchen after them, benefits, they get sick, leave work to have children etc.

 

[Tac 25]

replying to the first post of thread.

wow, that's a lot of profits.

more funds for Nvidia to develop better gpu's.

 

[Peter Ferrari]

replying to the first post of thread.

wow, that's a lot of profits.

more funds for Nvidia to develop better gpu's.

this will go to ai/datacenter r/d, not geforce development.

 

[bit_user]

I wouldn't be surprised if their manufacture volume strategy was to just keep up with demand so they don't fall further behind, but also don't necessarily catch up either.

Except I've periodically seen leaks about Nvidia having to sit on somewhat excessive inventories of high-end RTX 4000 GPUs, including even RTX 4090's, well after I'd assumed demand caught up with them.

 

[bit_user]

Like you said, r/d is part of it, so is software. But once you develop it and it pays off, it's like free to profit off it as long as you ca make it and sell it.

Jensen famously said "Nvidia is a software company". Their headcount is majority software and has a high burn-rate. They're continually adding new features, optimizations, and capabilities. Not to mention that a lot of the AI research isn't tied to any hardware generation, at all. Their software and AI R&D is primarily funded off the back of whatever is the hardware du jour.

The way you described it, it's as if you write an app from scratch and then basically stop working on it. Then, get rid of the deveolpers (except for a skeleton crew to do maintenance) and rake in the profits. That's not really how it works. Maybe they can make it look that way via accounting, by billing the software costs to the hardware that's recently-launched and in development, but most of their software is applicable to most of their hardware (from the last several generations).

 

[DSzymborski]

Meanwhile, a company-internal hotline, with actual humans providing such aid, could easily compete in costs with what this AI is supposed to be capable of. And in this context I wonder, whether someone in management skipped a cost-benefit analysis.

Well, can you provide data or is this just idle speculation?

 

[thestryker]

Except I've periodically seen leaks about Nvidia having to sit on somewhat excessive inventories of high-end RTX 4000 GPUs, including even RTX 4090's, well after I'd assumed demand caught up with them.

I'm talking about their H100 strategy. This generation of desktop graphics is screwed for a lot of reasons.

 

[Peter Ferrari]

Jensen famously said "Nvidia is a software company". Their headcount is majority software and has a high burn-rate. They're continually adding new features, optimizations, and capabilities. Not to mention that a lot of the AI research isn't tied to any hardware generation, at all. Their software and AI R&D is primarily funded off the back of whatever is the hardware du jour.

The way you described it, it's as if you write an app from scratch and then basically stop working on it. Then, get rid of the deveolpers (except for a skeleton crew to do maintenance) and rake in the profits. That's not really how it works. Maybe they can make it look that way via accounting, by billing the software costs to the hardware that's recently-launched and in development, but most of their software is applicable to most of their hardware (from the last several generations).

I'm no expert and was just paraphrasing it in terms of financial returns only. Stll, I think any software or r/d costs are irrelevant to the fact that they can earn 10x on every h100. what they do with the proceeds is another thing.

 

[bit_user]

That's irrelevant. The important thing is to discover the value of things. If somebody is willing to pay 10,000 for something that used to cost 100, that's a great discovery: the card is far more valuable than it was thought, hence when nvidia makes a lot more money from selling it, it can now buy more resources, with that profit, like more wafers, more time, better experts. Those have to be taken from alternative production, like gameboy consoles.

Wow, you've got that tied up so neatly, almost with a bow on it.

Except, is your memory so short that you forgot about the latest crypto boom? What happened to the value of consumer GPUs that was discovered? It wasn't durable, because it was driven by the value of crypto, which was in turn driven by hype. However, GPUs sold for enough and it lasted long enough that I think it convinced Nvidia and AMD to make bigger and more expensive GPUs, this generation, than they otherwise would've. That conclusion ultimately hurt them, their partners, and consumers.

We can also think back to the onset of the pandemic, when many switched to working & schooling from home. That created another bubble in PC hardware, driven by an external factor. Because so many PCs were sold at that time, abnormally few needed to upgrade during the past year, creating one of the worst periods for the PC sector in decades. Not only was the previously "discovered" value destroyed, but it's been a truly dire past year, for many in the PC industry.

Like those two examples, the current AI boom is generating outsized demand and has created something of a bidding war for the necessary hardware. The prices it's selling for might not be sustainable. I expect a lot of companies simply feel they cannot afford to miss this transition and be left behind. So, they might've decided to take a one time hit and make the investment to try and integrate AI, even though they have yet to fully establish a robust business case or RoI model for this kind of expense, going forward.

That's why I wouldn't be applauding Nvidia's pricing breakthrough, just yet ...unless you're a shareholder, I suppose. There's a lot yet to be written about the story of this AI boom, but it will have echos and implications on the hardware business for years to come, not to speak of the broader societal implications of AI, itself.

Because it has been discovered that nvidia cards are more valuable, society can spend limited resources making the most valuable stuff.

If all of humanity could make products that are worth 1000% than his cost, humanity would get super rich in no time.

Making a profit in the free market is being a philantrophe.

Capitalism is very good at shifting resources to where they tend to be most productive. However, let's not forget that this is ultimately fueled by human judgement, which is deeply flawed, and in numerous ways.

See also:

Homo economicus - Wikipedia

en.wikipedia.org