Google's Big Chip Unveil For Machine Learning: Tensor Processing Unit With 10x Better Efficiency

[Lucian Armasu]

Google announced that it has been working on designing a Tensor Processing Unit for the past few years, and has already used TPUs for over a year to power its applications and the AlphaGo AI.

Google's Big Chip Unveil For Machine Learning: Tensor Processing Unit With 10x Better Efficiency : Read more

 

[targetdrone]

Do you want skynet? Because that's how you get skynet!

 

[clonazepam]

Do you want skynet? Because that's how you get skynet!

Ooh I've got one.

But can it learn to play Crysis? And now we've gone full circle.

That'd be pretty cool to see Google, which was at one time nothing more than a search engine... develop everything (I was going to do a list but laziness and 'everything' seems plausible at this point).

 

[ZolaIII]

Anyone with at least litle brains would see how ASICS aren't a way to go into the future. Couple of reasons to begin with are; fixed logic (not programmable) in conjunction to absence of perfected algorithms & as much as days of specialized accelerators are coming it would still be a very unefficient (power/cost/die size) to have one of those for each & every task. Compared to last actual gen of ADSP's this is really just a little more efficient (cost or performance wise) on the other hand ASICS will always be 2 steps behind; lagging a gen or two behind in implementing latest algorithms, not very good solution because they can't be upgraded. ASICS are OK but for simple fixed function logic that won't need any future development. They certainly aren't for something that should be flexible as it can & that should be able to self evolve. Curent so cold deep machine learning AI is on the level of reflex behavior & that certainly isn't intelligence. In order for it to be able to evolve it needs fully opened pathways. Only suitable hardware platform for it their for is a large area (of programmable gates) FPGA in it's modern design that actually integrate & ADSP (with SIMD blok) & that is fully reprogram-able & capable of simultaneously multiple tasks execution. I won't even waist words how much GPU's aren't suitable for this task.
Google's fat bottom girls are getting really lazy & if they continue like this they won't last very much longer.

 

[sh4dow83]

Am I the only one who doesn't want machines to learn?

 

[wifiburger]

this seems viable for next gen gaming, with a bit modification we could have some interesting learning AI without wasting cpu time,

sony should integrate something similar in their future playstation, I know I would buy it

 

[kenjitamura]

and there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold.

Putting that in there is just plain cruel. We all know that Google will not be making CPU's to challenge AMD or Intel in the foreseeable future.

 

[bit_user]

Anyone with at least litle brains would see how ASICS aren't a way to go into the future.

Do you really think Google doesn't know what they're doing? Their blog post said they have over 100 teams that are using machine learning. This is a problem they understand very well, and I'm sure they calculated the economic benefit of having such an ASIC, before they invested the millions of $'s needed to design, manufacture, and deploy it.

And because they're Google, they can make one ASIC that's designed to accelerate today's workloads. As soon as that's debugged and starting to roll out, they can begin designing another ASIC to tackle the next generation of machine learning problems. That's how it "evolves" to meet future needs.

 

[jamiefearon]

Is it possible to turn this into a PCI Express expansion card, that will be work with current PC's?

 

[bit_user]

and there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold.

Putting that in there is just plain cruel. We all know that Google will not be making CPU's to challenge AMD or Intel in the foreseeable future.

Definitely agree. Lucian is extrapolating unreasonably, here. Designing an ASIC to calculate with tensors is far simpler than developing a leading-edge CPU. And I think it'd be really hard to do it with sufficiently better perf/Watt to justify the investment over just partnering with Intel/AMD/Qualcomm/etc. I mean, we're talking hundreds of $M, to design a single CPU. Are the margins that high on existing CPUs, for a huge customer like Google?

And the article is potentially off the mark on another point: when Xeon D was launched, I read a detailed article (on another site) about how Intel worked with Facebook to create the Xeon D. The problem they had was that the perf/Watt of the E5 Xeon series wasn't improving at the same rates as Facebook's workload was increasing. And the Silvermont Atom-based server CPUs were slow enough that they exceeded FB's latency requirements (ARM: are you listening?). So, they basically power-optimized a multi-core Broadwell.

I'm sure Intel also talked to Google about it, but I don't know if it was the same level of collaboration as they had with FB.

 

[bit_user]

Is it possible to turn this into a PCI Express expansion card, that will be work with current PC's?

I think google would rather sell you time on their TPU-equipped servers in the cloud, than get in the business of selling hardware to end-users.

For hardware in your PC, GPU's aren't bad. This year, Intel is supposed to introduce a line of Xeons with built-in FPGAs that could be configured to do something comparable to (though maybe smaller than) Google's ASIC. I'm hoping that capability will trickle down to consumer CPUs in coming generations.

I'd love to know the size of that ASIC. I wouldn't be surprised if it were really big and low-clocked, because they can afford a higher up-front manufacturing cost to offset lower lifetime operating costs.

 

[ZolaIII]

Anyone with at least litle brains would see how ASICS aren't a way to go into the future.

Do you really think Google doesn't know what they're doing? Their blog post said they have over 100 teams that are using machine learning. This is a problem they understand very well, and I'm sure they calculated the economic benefit of having such an ASIC, before they invested the millions of $'s needed to design, manufacture, and deploy it.

And because they're Google, they can make one ASIC that's designed to accelerate today's workloads. As soon as that's debugged and starting to roll out, they can begin designing another ASIC to tackle the next generation of machine learning problems. That's how it "evolves" to meet future needs.

Google don't actually give shit about it, they will go with best possible solution that is available to them as licensable IP that someone else is ready to put on the silicone by as loo price as possible.
Braking news for you: Google didn't developed nor it's producing this ASICS! They adapted it for usage. If you think Google develop their current neural network pathways into algorithms by using ASICS that ou boy you are even dumber than you look.
Another braking news for you Google is still far behind IBM when it comes to AI in both application usage & real understanding of it as they started developing it much after IBM did. They naturally did it using large aria FPGA's.
If you actually consider even usual tasks that are offloading by usage of accelerators today; vision processing, video & audio processing, cryptography... & to just add deep learning among many more to come you will (finally) see what I am talking about. Single ASICS is cheaper & more power efficient (but not by large margins) then ADSP & more compared to FPGA's but they are certainly more expensive & less power efficient when you add one for every task I mentioned & their are many more uses to come + when you add into consideration that when ever their is a lap forward (by usage of better logic) you will actually have to make new ASICS (& make peple buy it) which costs much (for manufacturer to develop & user to buy) wile taking at least one year to be manufactured & deployed as final product... I hope that you at least now see why & how their aren't suitable (& will never be).
So for this to live up certain base hardware expectations are needed to be closely followed up basically a complex SoC containing general purpose cores (CPU's), general purpose accelerators like GPU, special purpose defined fixed logic (ASICS for simple tasks as scheduling), all around suitable accelerator (FPGA's with DSP's).
We can't learn anyone anything especially not machines if we act plane dumb.

I am disappointed with level of intelligence of peple that post hire this day's.

 

[PaulAlcorn]

and there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold.

Putting that in there is just plain cruel. We all know that Google will not be making CPU's to challenge AMD or Intel in the foreseeable future.

Definitely agree. Lucian is extrapolating unreasonably, here. Designing an ASIC to calculate with tensors is far simpler than developing a leading-edge CPU. And I think it'd be really hard to do it with sufficiently better perf/Watt to justify the investment over just partnering with Intel/AMD/Qualcomm/etc. I mean, we're talking hundreds of $M, to design a single CPU. Are the margins that high on existing CPUs, for a huge customer like Google?

And the article is potentially off the mark on another point: when Xeon D was launched, I read a detailed article (on another site) about how Intel worked with Facebook to create the Xeon D. The problem they had was that the perf/Watt of the E5 Xeon series wasn't improving at the same rates as Facebook's workload was increasing. And the Silvermont Atom-based server CPUs were slow enough that they exceeded FB's latency requirements (ARM: are you listening?). So, they basically power-optimized a multi-core Broadwell.

I'm sure Intel also talked to Google about it, but I don't know if it was the same level of collaboration as they had with FB.

The article states that "there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold." There are, in fact, long running rumors. There are hundreds (if not thousands) of articles on the subject that can be found dating back to 2013. If Google were to build its own chips it would likely not compete with Intel in the open market, but instead use them for its own purposes.

Intel stated in 2014 that it had 18 customers it created custom chips for, including all of its 'teir one' customers. These included 15 custom designs, and the company indicated that it planned to double that number in 2015. It also indicated that the orders range from tens of thousands to hundreds of thousands of units, so it is certainly doing a decent business with it.

 

[bit_user]

The article states that "there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold." There are, in fact, long running rumors. There are hundreds (if not thousands) of articles on the subject that can be found dating back to 2013. If Google were to build its own chips it would likely not compete with Intel in the open market, but instead use them for its own purposes.

Right. I've definitely seen stuff, to that effect. But how much of that is just wild speculation, based on their hiring of chip designers vs. actual leaks about what they were doing?

To me, it seems likely that they're building specialized chips that either don't exist on the market, or are very closely matched to Google's internal needs & software. Exactly like this TPU.

I think the likelihood of them building a general-purpose CPU, that would be competitive with something like a Xeon, is near zero. If they could succeed, it wouldn't be by a very big margin, and the cost would be so high that it's difficult to see them amortizing it over the useful lifespan of the chip (which would only be another generation or so of x86 CPUs).

Intel has tuned their x86 chips to the point that you can't possibly beat them by doing only a few things better. You have to do everything well, and a few things extremely well. And this could take generations to accomplish, by which point the entire fabrication process of CPUs might be in upheaval, as lithography hits a wall. So, the only way it makes sense to DIY is if your workload is drastically different than the target of a typical server CPU or GPU.

 

[bit_user]

Braking news for you: Google didn't developed nor it's producing this ASICS! They adapted it for usage. If you think Google develop their current neural network pathways into algorithms by using ASICS that ou boy you are even dumber than you look.

If you care more about facts than opinions, I'd encourage you to check out some articles about this TPU on eetimes.com. I'm normally reluctant to refer to another site, but it does go a bit more in depth on the design team and their rationale for building a custom ASIC.

On the other hand, if you're just trying to hype up FPGAs and IBM because of stock shares you own, you'll be happy to know that I have no intention of engaging in a point-by-point, back-and-forth refutation with you. I have better things to do with my time, and you'd be well served by spending yours on such things as improving your spelling & grammar, and learning how to debate issues without resorting to childish insults.

We can't learn anyone anything especially not machines if we act plane dumb.

I am disappointed with level of intelligence of peple that post hire this day's.

Well said.

 

[PaulAlcorn]

The article states that "there are long-running rumors that Google is developing its own CPU to break the Intel stranglehold." There are, in fact, long running rumors. There are hundreds (if not thousands) of articles on the subject that can be found dating back to 2013. If Google were to build its own chips it would likely not compete with Intel in the open market, but instead use them for its own purposes.

Right. I've definitely seen stuff, to that effect. But how much of that is just wild speculation, based on their hiring of chip designers vs. actual leaks about what they were doing?

To me, it seems likely that they're building specialized chips that either don't exist on the market, or are very closely matched to Google's internal needs & software. Exactly like this TPU.

I think the likelihood of them building a general-purpose CPU, that would be competitive with something like a Xeon, is near zero. If they could succeed, it wouldn't be by a very big margin, and the cost would be so high that it's difficult to see them amortizing it over the useful lifespan of the chip (which would only be another generation or so of x86 CPUs).

Intel has tuned their x86 chips to the point that you can't possibly beat them by doing only a few things better. You have to do everything well, and a few things extremely well. And this could take generations to accomplish, by which point the entire fabrication process of CPUs might be in upheaval, as lithography hits a wall. So, the only way it makes sense to DIY is if your workload is drastically different than the target of a typical server CPU or GPU.

I agree that it might appear to be hard to amortize the cost over the initial lifespan of the chip, but it would give them the advantage of a foothold into future chips, which will become cheaper with time. Google is Intel's #1 customer, consuming somewhere on the order of 1.2 million CPUs per year. As such, they are surely getting a steep discount, which the incessant rumors probably help push along.

If we were to assume a price of $1,250 per CPU, which is a low estimate in my opinion, that is $1.5 billion in annual spend. Yes, developing its own CPU is expensive up front, but they are already shelling out plenty. Also, Google's datacenter YoY growth rate is roughly 10 to 20 percent, depending upon which analyst you choose to believe, so a future of its own chips might seem even more attractive in the long run.

Yes, hiring semiconductor engineers is the red flag that started much of the talk, but there is also the Google 'build it yourself' ethos. Google isn't very keen to share, unlike the FB and OCP movements, so it always remains shrouded in secrecy, which adds more fuel to the rumor fire. (Google just joined OCP this year, and is only contributing a power delivery and rack system, at least at last check.)

If Google were to roll its own silicon I am sure that it would not be a general purpose CPU, nothing they build is "general”, everything is as optimized and customized as possible - hyperscale at its finest. It would never be sent to the general market, that much is for sure.

I am unsure whether or not to believe the rumors, but I would certainly not be surprised. it bought into robot killer attack dogs (at one point), so why not?

 

[bit_user]

If Google were to roll its own silicon I am sure that it would not be a general purpose CPU, nothing they build is "general”, everything is as optimized and customized as possible - hyperscale at its finest. It would never be sent to the general market, that much is for sure.

By exotic, I had in mind something more like an integer-only CPU core, so they could pack huge numbers of them on die and eliminate the drain of FPUs for workloads that don't need them. That's the best I can come up with, at the moment, but I think there'd have to be something weird about it, that puts it well beyond the reach of any current, general-purpose CPU.

Their "order of magnitude" claim about the TPU is probably a good benchmark for what their future chips should try to achieve.

Anyway, thanks for the data cited in your post. I had some arbitrary numbers bouncing around in my head, but real data earns my up-vote.

it bought into robot killer attack dogs (at one point), so why not?

I'm sure that acquisition was for the tech, and probably because someone put together a high-level strategy around the robotics market that required it and justified the cost. I'm no Google-expert, but companies don't just spend that kind of money (not to mention splitting their focus) on a whim.

I think we agree on most points. Since I'm just speculating, I'll probably leave it at that.

 

[bit_user]

So, it occurred to me that what might underlie Google's DIY ethos is a deep, abiding belief that they can do it better.

I see two consequences of this: first, when the opportunity for success is small, they might be less likely to try. Second, when they actually try something and it's not better, we're unlikely ever to hear about it.

 

[Sitarow]

ASIC chips paired with specified algorithms are a way to departmentalize specific algorithms into sections of hardware (ASIC) that support higher functioning process.

"Much like how unconsciously, breathing is controlled by specialized centers in the brain stem, which automatically regulates the rate and depth of breathing depending on the body’s needs at any time."