News Cerebras video shows AI writing code 75x faster than world's fastest AI GPU cloud — world's largest chip beats AWS's fastest in head-to-head compar...

[Admin]

Llama 3.1 405B runs at nearly a thousand tokens a second on Cerebras Inference, and took a quarter of a second to get the first token.

Cerebras video shows AI writing code 75x faster than world's fastest AI GPU cloud — world's largest chip beats AWS's fastest in head-to-head compar... : Read more

 

[bit_user]

Pretty impressive numbers, but I find it interesting that cost wasn't mentioned once.

While we can't exactly know what even hardware with an actual list price would cost big customers, we can simply look at the hourly cost for running those models on each of these services. I'd like to see that comparison!