They did it for power efficiency. I think if you look at Intel's endgame with this architecture, they want to be more like AMD with their chiplet designs. Moving more things off-die and focusing on power efficiency will allow them to make their server processors more like Epyc where they can use regular desktop dies to make server chips. Also separating the memory controller and other functions into tiles again will make this transition easier.
I don't think this has anything to do with power efficiency. It was really just a cost saving measure IMO. The IO tile uses TSMC N6, and since external interfaces (read: memory controllers) don't scale well with smaller process nodes, putting that on the most expensive compute tile (N3B) would have increased costs quite a bit. I mean, not like MASSIVELY, but for bean counters it might have been 5% more expensive overall. That's a big change.
So, really, I
do know why (I think) Intel put the memory controller on the IO tile. I just think it was a very odd and shortsighted decision. And in a similar vein, and I've talked with Paul about this as well, it seems absolutely crazy that Intel hasn't opted to put some sort of large cache tile into the mix here. The engineers at Intel have known about the gaming benefits of large caches for about ten years: Broadwell i7-5775C. That thing rocked for gaming, often beating the "superior" Skylake chips. But it was clocked lower and never really intended to be the next big thing.
AMD took the idea and ran with it on Zen 3 X3D, with major benefits. Intel could easily (relatively) do the same thing with the current tiled architectures. Link up one more chip somewhere, a big fat L3 cache of 64MB, and suddenly you have a gaming contender again. My bet? Intel will do this with Nova Lake. It would be foolish not to do it by then, so maybe we could even see an Arrow Lake Refresh that adds a cache tile.
In the grand scheme of CPU architectural design, this should be pretty trivial to do compared to all the stuff that's happened with cores and threads and decode/execute instruction widths. Lion Cove is 8-wide compared to the 6-wide used for the past eight years. I seriously don't get how no one felt adding a bigger cache was justifiable.
AMD is killing Intel on price in the server market right now. Last quarter they sold more server revenue than Intel, which has Intel panicking. If Intel can use their desktop dies to make server chips, they could lower their prices enough to compete again with AMD. The current Intel generation gave zero f's about gaming. They are positioning themselves for their next server products coming in the future. Mark my words. We will see Intel server chips made from desktop dies in the future.
I agree with the rest, though. Intel needs tiled architecture server chips, three years ago. Again, crazy that it hasn't happened already! Not that Ponte Vecchio didn't try (and have plenty of other issues... but that's a GPU, not a CPU).