News "Everyone and Their Dog is Buying GPUs," Musk Says as AI Startup Details Emerge

[edzieba]

For me, what's surprising is that he's not using Tesla's Dojo supercomputer. I wonder if that's because it's not as good at Transformer networks, or just because they can't build it up fast enough to accommodate the additional demand.

Dojo is designed for training a optical recognition and decision making system from captured camera and synthesised renders of views from a moving vehicle. It was not designed for efficiently training a language model or some other network type. On top of that, Dojo is busy training an existing network, so retasking it would slow progress on that.

training an AI with twitter's data ? that would be the perfect recipe for a Sociopath skynet lol
Twitter is a cesspool of horrible, nasty people. Even worse since EM reversed the ban on a lot of offensive and fake news accounts
Elon, please use 4chan's data too to complete the model, while you are at it.

GPT-3 and GPT-4 (among others) are already trained on the CommonCrawl dataset, which contains text scraped from Twitter (along with many other sources).

Being stinking rich doesn't really make you intelligent, though. Likewise, being intelligent does not mean you are a scientist. He is a businessman. He doesn't know a hoot about rockets or satellites, so please don't act as if he did anything else than spending money for their development because he didn't. He didn't develop jack. And with Twitter, he also proves that you can still be a clown even when you are a successful businessman.

Not according to people who work for SpaceX (e.g. Tom Mueller, designer of the Merlin engine) or interacts with at NASA, who would not be so easily fooled.
The problem is he's an arsehole, and it's an attractively poetic notion that arseholes cannot also be competent. That is clearly not the case here. He's not infallible - Twitter's best utility would be in immediately ceasing its own existence - but is far from not knowing how rockets work.

:EDIT:: As someone once put it: CTO Elon knows what they're doing. CEO Elon needs to be slapped on a daily basis.

 

[trevorno]

Is this company a separate entity? Not sure if it was mentioned in the article.

 

[bit_user]

Dojo is designed for training a optical recognition and decision making system from captured camera and synthesised renders of views from a moving vehicle. It was not designed for efficiently training a language model or some other network type.

Okay, so you have actual knowledge that it's inefficient at training transformers? Or is this more speculation phrased as facts?

On top of that, Dojo is busy training an existing network, so retasking it would slow progress on that.

Obviously. But, you'd imagine they might be able to build some more instances in possibly less time than it takes Nvidia to work through their order backlog.

 

[russell_john]

You're badly misinformed about this stuff.

I'm not quite sure I follow what you're saying about 10k H100 = 80k AD100. As far as I can tell, there exists no such chip as the AD100. There's an A100, which is made on TSMC N7. However, the AD102 (RTX 4090) and H100 are both made on TSMC 4N. And the GA102 used Samsung 8 nm.

As for the AD102 vs. H100, the former die is 608.5 mm^2 in size, while the latter is 814 mm^2. And no, the AD102 isn't merely shaved down from the H100 - they're fundamentally different designs. The H100 lacks a display controller and (virtually?) any raster or ray-tracing hardware. The H100 also features vector fp64 support, which the AD102 lacks, and replaces GDDR6 memory controllers with HBM2e or HBM3.

We really haven't seen Nvidia repurpose a 100-series GPU for "gaming" cards since the Titan V launched in 2017.

Here's a secret decoder ring. Study it. Learn it. Know it.

​

https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units​

​

The conflict exists at the wafer level. Nvidia ordered a certain number of 4N wafers from TSMC. Until those wafers go into production, they can choose which mask to use on them. So, it's diverting wafers instead of chips. But, the tension is real.


Whoever is saying that isn't looking at their quarterly reports. Their revenues are still down from their peak (as of the latest quarter to end), but they haven't been unprofitable for a long time.


If it's truly RX 6000 GPU orders that AMD is slashing, then no. Most of those are made on TSMC N7, and the 4N process Nvidia uses for AD GPUs requires entirely different fab production lines. I don't even know if the N5 and 4N production can share much/any of the same equipment.


That's basically what the article said, but I think $35k isn't even the "normal" price for H100's. If you bought back in November or so, they'd probably have cost much less. I seem to recall seeing a price of $18k, but I'm not 100% sure that wasn't for an A100.

Indeed $18,000 is for the A100 ..... Nvidia suggested price for the H100 is $33,000 - $38,000 depending on the configuration ... If I remember right it as something to do with either the amount or speed of the HBM3 memory it has

 

[jkflipflop98]

Ever heard of Howard Hughes?

The line between greatness and insanity is not always so clear and bright. Someone could also start on one side of it and veer onto the other. Potentially adding mind-altering drugs to that mix could make for a very volatile combination.


I don't know how closely you've followed the Twitter drama, but Elon has yet to demonstrate much in the way of genius or even competence, at the helm of it. A lot of his decisions seem fairly rash and often have to be walked back. And he's put some rather fundamental policy decisions to a twitter poll, as if on a whim.

I'm not a genius, but I work with them everyday. I understand fully that genius and madness are two sides of the same coin.

The thing us "old folks" don't understand about Elon is his sense of humor. I spend lots of time browsing r/dankmemes so I get it. I get that he's just making jokes and his bullhorn just so happens to be louder than anyone elses. He happens to be a very valuable human who's comments get taken more seriously and misunderstood by basically anyone over the age of 16. Its so funny when he's obviously just posting a meme and the talking heads are dissecting it and trying to figure out what it means for Tesla or SpaceX's future. It doesn't mean anything. It's a stupid joke.

When a robot asks me for my estimated run time on a machine, my default answer is 69,420 minutes. Is it a juvenile and sophomoric thing to laugh at? Sure, but you have to laugh sometimes and it's doubly funny when you're in such a sterile and high-precision environment - and your login credentials identify you as a "Level 55 Wizard". The dumb basicness of the idea contrasted with that life is what makes it hilarious.

And "mind altering drugs"?? Please. He hit a joint a few times on a podcast once. I know true blue geniuses that eat a bag of shrooms every weekend then come back to work Monday morning and make magic happen. Don't believe everything Dick Nixon tells you.

 

[Deleted member 14196]

Elon is an engineer, genius or not the man is really good at finding funding for his projects. I think that’s his strong point.

Also, as noted, he does have a sense of humor. I recall seeing on one of the SpaceX vehicles, one time a reference to the movie top secret.

Take a look at Tesla, making money, hand over fist, where everybody else is failing hard. Legacy car manufacturers will probably go out of business. Because they can’t get their heads around the new processes that Tesla uses to reduce production costs. Their next car is going to slash production cost by 50% and this will allow them to sell a $25,000 vehicle and still make a healthy profit. No other car company can do that. None. The man is not stupid and don’t underestimate him.

Stupid General Motors and other OEM car manufacturers couldn’t be talked into using giga-castings, well now all that’s going to change in a big big way because of Tesla. I predict that the existing legacy carmakers may exist for a while, making ice vehicles, but they’re not going to be very competitive in the EV market. In fact, they can’t because their processes are so antiquated and stupid and wasteful, and they are so proud and knows so much about making cars that they will probably not be able to make the transition from where they are today to a completely software designed vehicle like Tesla has

SpaceX is also hugely successful, and will only become more and more successful. Look at the track record to date. Excellent. Why is it that no other company can drive down the costs of space travel but they can? And the answer is because they’re not stupid and they don’t think inside the box. they do things that engineers would do for efficiency not that unqualified financial guys do to save money.

 

[lmcnabney]

Elon is an engineer, genius or not the man is really good at finding funding for his projects. I think that’s his strong point.
Take a look at Tesla, making money, hand over fist, where everybody else is failing hard.
SpaceX is also hugely successful, and will only become more and more successful.

Tesla owes their fortune to fat EV tax credits and SpaceX is funded by governments.

Elon Musk - Welfare queen.

 

[Deleted member 14196]

Like I said, his strength is getting others to spend their money. That’s what he excels at. However, those are two of the most successful ventures he’s been in.

 

[InvalidError]

Tesla owes their fortune to fat EV tax credits and SpaceX is funded by governments.

Elon Musk - Welfare queen.

Don't forget selling green credits.

 

[Why_Me]

Elon the grifter. Trying to use his influence to artificially raise the prices of GPUs. Pretty sad attempt.

I sure hope the 'W' in your moniker stands for Wisconsin and not Washington.