News Elon Musk Buys Thousands of GPUs for Twitter's Generative AI Project

[Admin]

Musk reportedly kicks off generative AI project at Twitter, ups compute horsepower.

Elon Musk Buys Thousands of GPUs for Twitter's Generative AI Project : Read more

 

[PlaneInTheSky]

AI is the next scam.

After decades of Tesla using "AI" and "machine learning", the cars are still horrendous at self-parking. Companies not relying on "AI" for parking have been able to properly park for years.

 

[bniknafs9]

am i dumb or is AI the next Gen business , after Mining.

 

[JamesJones44]

am i dumb or is AI the next Gen business , after Mining.

Yes and no. The word AI has definitely become the hot keyword, just simply put AI in your company name, throw it out in a conference call or say your doing something (anything) with it and you will get a stock pop. Most companies that claim they are doing AI work aren't or are just simply leveraging existing ML tools to extend some existing functionality than doing anything that will move the needle sales wise.

However, AI in computer vision has been a big move forward and AI in the form of LLM has also been able to do things that were difficult to do with lines of code (not impossible, but difficult and expensive). In this way AI is a next gen business, but with some caveats. However, companies jamming LLM into every product and calling it a revolution, well, that is largely just cashing in on the hype.

In my personal opinion the companies that best leverage AI will be the winners, but I don't think their will be a single company that will get associated with AI the way things like search has with Google or Netflix with streaming. People are trying to do it with ChatGPT, but ChatGPT is just one type of AI, there are many many forms of AI/ML that are good at specific tasks and I think the idea of one model to rule them all is unlikely at this point in the cycle/evolution.

 

[hotaru251]

AI is the next scam.

its not.

ai has uses just needs to mature properly.


especially in fields like medical where theres a ton of info and having an ai use your given input to offer outputs can save you time.
same for coding as bug testing your code is timely and ai doing it for you quickly is helpful.


whatever reason Musk wants it for thoguh is for sure a waste of money, effort, & power.

 

[jkflipflop98]

AI properly applied is most certainly no scam. We've used it at work for years for all kinds of tasks.

It's one of the next big steps in computing. We have insane amounts of compute power and access to almost all knowledge of the combined human race. It makes logical sense to use the former to exploit the latter and come up with trends and points and combinations that a human would have never figured out.

 

[InvalidError]

AI is the next scam.

I wouldn't say it is a scam but the AI-ification of some relatively simple things is just plain stupid. Parking is basic geometry. Using AI to find the spot, make sure it is safe to get in and monitor progress makes sense. Once all the necessary distances have been measured though, parking itself should be a pre-programmed sequence as long as tires don't slip and surrounding parked cars stay put.

 

[evdjj3j]

its not.

ai has uses just needs to mature properly.


especially in fields like medical where theres a ton of info and having an ai use your given input to offer outputs can save you time.
same for coding as bug testing your code is timely and ai doing it for you quickly is helpful.


whatever reason Musk wants it for thoguh is for sure a waste of money, effort, & power.

I would love a service that I could upload an MRI or some other diagnostic imaging to and have an AI double check the human radiologist.

 

[domih]

I would love a service that I could upload an MRI or some other diagnostic imaging to and have an AI double check the human radiologist.

It is already an industry which has been active for several years and is in full expansion. See for instance, among others, https://deepbio.co.kr/ or https://www.gleamer.ai/. There are 100+ hardware and software companies, using AI, working in radiology or pathology, etc.

There are already quite a lot of hospitals or other health organizations using these solutions on site.

The practitioners (e.g. radiologists) are not replaced by AI. They still are the ones interpreting the X-rays, CAT-scans, etc and writing the reports. The AI solutions are only helping by detecting issues that a human might miss. This is quite important in the case of cancers where the earlier the detection is the better it is to fight it.

The NY Times and many other mainstream press have many articles about this industry. It does not make noise like ChatPGT but it is a serious business with massive financial weight.

You just have to look for it to find it.

You can also look for DICOM, a file format and a protocol used in radiology, dentistry, etc. If you see an X-Ray on a doctor or dentist monitor, it is probably a DICOM file.

HTH

 

[russell_john]

We need to stop training AI (So I can catch up)

- Elon Musk

 

[bit_user]

Hmm... Musk has business relationships that pose some interesting questions.

1. As one of the OpenAI founders, can he no longer gain access to their technology, or does Microsoft now control too much of the company for him to have such pull?
2. Considering Tesla's AI hardware sounds pretty impressive, why not arrange to buy some of theirs? Are there technological differences that significantly disadvantage it on running transformer networks?

When big companies like Twitter buy hardware, they buy at special rates as they procure thousands of units.

Yes... but, Nvidia is basically the single supplier of the hardware everyone wants to use. That gives them quite a bit of leverage, in any price negotiations.

10k GPUs is a lot of money for a company that (I think) is still making losses. If we assume about $20k each (including the servers to host them), that's a cool $200M. Only about 1% of what he paid for Twitter, but probably a multiple of Twitter's annual hardware spend.

Nvidia's H100 boards can cost north of $10,000 per unit,

Somehow, I had a figure of $18k in mind. Not sure if I'm misremembering that or maybe the street price has shot way up since then. shopping.google.com shows prices anywhere from $28.5k to ebay prices of $43k or more.

Then, I thought I'd see what Dell's list price is, so I popped over to dell.com and looked at the price of adding one to a PowerEdge R750xa. They want an absolutely astounding $86,250 per H100 PCIe card, and they make you add a minimum of 2 GPUs to the chassis!!! Having a decent amount of experience with Dell servers at my job, I know they like big markups for add-ons, but I'm still pretty stunned by that one.

If you know anything about these, you're probably aware that the PCIe cards aren't even the best type of H100. What you really want are the SXM version. And a further irony is that a pair of the current H100's cannot even run GPT-3, which is why Nvidia recently announced a refresh of its H100 with more memory, due out in Q3.

 

[bit_user]

I don't think their will be a single company that will get associated with AI the way things like search has with Google or Netflix with streaming.

At the hardware level, Nvidia currently seems to dominate the market for server-based AI. They've been trying to use their software as further leverage, including by providing lots of tools, libraries, and pre-trained models.

 

[eldakka1]

I hope the seller demanded cash payment up front ...

 

[zodiacfml]

Should have been done under Tesla which can burn cash, has a lot of free electricity, and there's already an AI team there.

Musk reportedly kicks off generative AI project at Twitter, ups compute horsepower.

Elon Musk Buys Thousands of GPUs for Twitter's Generative AI Project : Read more

 

[froggx]

I wouldn't say it is a scam but the AI-ification of some relatively simple things is just plain stupid. Parking is basic geometry. Using AI to find the spot, make sure it is safe to get in and monitor progress makes sense. Once all the necessary distances have been measured though, parking itself should be a pre-programmed sequence as long as tires don't slip and surrounding parked cars stay put.

humans have been parking since the dawn of time and always don't do too horribly sometimes. actual AI parking is prolly more for hard to judge things like to pop the handbrake at speed and 180 into a too small slot during a blizzard tornado.

 

[Endymio]

After decades of Tesla using "AI" and "machine learning", the cars are still horrendous at self-parking.

Decades? Tesla's first AI-based Autopilot release was in 2014, and your video tested a Tesla made in 2018 (2019 model year). Shortly after that, Tesla began replacing its ultrasonic-based Autopark with a vision-based system.

 

[JamesJones44]

At the hardware level, Nvidia currently seems to dominate the market for server-based AI. They've been trying to use their software as further leverage, including by providing lots of tools, libraries, and pre-trained models.

Yes, I 100% agree from a model training and execution perspective Nvidia is and looks like it will be the big winner. However, from a consumer standpoint I don't think it will be synonyms with a single source.

 

[bit_user]

Decades? Tesla's first AI-based Autopilot release was in 2014, and your video tested a Tesla made in 2018 (2019 model year). Shortly after that, Tesla began replacing its ultrasonic-based Autopark with a vision-based system.

Thanks for checking all of that. I was suspicious the video or cars were outdated. That classifies it as misinformation, which should be reported.

 

[bigdragon]

The practitioners (e.g. radiologists) are not replaced by AI. They still are the ones interpreting the X-rays, CAT-scans, etc and writing the reports. The AI solutions are only helping by detecting issues that a human might miss. This is quite important in the case of cancers where the earlier the detection is the better it is to fight it.

This is the kind of AI I think most people can like and support -- AI that doesn't replace humans but instead cooperatively augments human capabilities and analysis. The human tech and the AI work together to improve outcomes.

A lot of the people who are against AI or call it a scam focus on the AI that steals from and replaces people. Stable Diffusion and ChatGPT make news for disruptively replacing artists and writers.

Cooperative AI doesn't seem to get amplified the way thieving/disruptive AI does.

 

[JamesJones44]

1. As one of the OpenAI founders, can he no longer gain access to their technology, or does Microsoft now control too much of the company for him to have such pull?
2. Considering Tesla's AI hardware sounds pretty impressive, why not arrange to buy some of theirs? Are there technological differences that significantly disadvantage it on running transformer networks?

Per my understanding Musk was just a seed investor in the non-profit portion of OpenAI. Barron's made it sound like OpenAI's non-profit and for profit funding are subdivided, if this is the case Musk likely doesn't have a stake in the for profit portion directly (non-profit OpenAI Inc controls 51% the for profit OpenAI LP, so indirectly he does have a stake per say). Per that same Article, the MS's investment is a non controlling stake (49%), so I don't think MS would be able to block his access to anything from the company to Twitter, though it's hard to say if there is an exclusivity clause tied to the MS investment. MS gets 75% of OpenAI's profit until it recoups the initial investment, so with that in mind it seems like an exclusivity deal would be at odds with that part of the deal, but without full details of the contract it's just speculation.

For the 2nd point I wondered the same thing. At a minimum they could use AWS EC2 instances with a cluster of P4's and then switch when the product is ready to roll. For a cash strapped company (per Musk's claims) it seems like a more sensible option.

 

[bit_user]

This is the kind of AI I think most people can like and support -- AI that doesn't replace humans but instead cooperatively augments human capabilities and analysis. The human tech and the AI work together to improve outcomes.

There's no way this isn't putting radiologists out of a job, in the medium or longer term.

 

[bit_user]

At a minimum they could use AWS EC2 instances with a cluster of P4's and then switch when the product is ready to roll.

Oh no. That's useless for training, which is obviously the immediate focus.

For inference, you need enough aggregate GPU memory to hold the entire network, which P4's certainly don't have. I think if you had an instance with like 4x GPUs with 48 GB each, that would be enough to inference a GPT-3 class network.

 

[JamesJones44]

Oh no. That's useless for training, which is obviously the immediate focus.

For inference, you need enough aggregate GPU memory to hold the entire network, which P4's certainly don't have. I think if you had an instance with like 4x GPUs with 48 GB each, that would be enough to inference a GPT-3 class network.

You can cluster them using P4ds and aggregate the memory that way, we've done it a few times when hardware was limited. It's much slower than running on direct hardware, but you also don't need to put down 10,000 a pop for the set of GPUs needed for the full network.

 

[bit_user]

You can cluster them using P4ds and aggregate the memory that way, we've done it a few times when hardware was limited. It's much slower than running on direct hardware, but you also don't need to put down 10,000 a pop for the set of GPUs needed for the full network.

I don't know how many parameters you were trying to train, but an estimate I've seen is that it took about 34 days to train GPT-3 on 1k Nvidia A100's. With a workload of that scale, I think piddly P4's won't get you anywhere.

 

[JamesJones44]

I don't know how many parameters you were trying to train, but an estimate I've seen is that it took about 34 days to train GPT-3 on 1k Nvidia A100's. With a workload of that scale, I think piddly P4's won't get you anywhere.

A single one, no, but cluster of them will though. You can look up what Amazon coins as an "UltraClusters" for an example. That is not what we use to chain them together, but it's an example of what can be done with them to achieve a large array of GPUs to use in model training.