I don't think a L3 cache miss combined with a L4 cache hit will cost the tCPU close to 100 clocks. If thats the case, Intel may forego ADM L4 altogether! I'm think it's gonna be in the ballpark of X3D Vcache. A few dozen cycles at most.
The reason why AMD's V-cache doesn't add latency to on-die L3$ is because the the V-cache is a straight-up space extension to on-die L3$ and shares the on-die L3$ tag-RAM which is where most of the latency comes from - it takes time to scan a large tag-RAM for an address match.
Makes sense as latency in the range of 100 clock cycles doesn't make sense for the tCPU. But acting as L3 for the tGPU makes a lot of sense as it might be able to service the tGPU's L2 within 30 - 50 clocks. Might be worth it after all!
That sounds much more intersting to me, too. Actually I could even see benefits of a scratch pad area that isn't even mapped to RAM (or has a relaxed coherency and snoop requirements) and for exclusive use by say the iGPU or a neural accelerator.
These are all things that intel already works on for their server xeon CPUmax although I do think that this cache will be transparent and just show up as a big cache that every software that can use a big cache will be able to use.
Just to clarify, when I said "not mapped to RAM", I meant not with DRAM backing, it obviously would have to be memory mapped because they wouldn't want to create an ISA extension just for that.
Intel's patent lists about a dozen non-cache uses for the thing. It'll likely get partitioned between multiple things from IGP caching to kernel scratchpad and high-security memory for DRM, TPM, etc.
I'm sure that it will be partitioned for all of these things, that doesn't mean that there will be none left over to act as a basic cache.
While it doesn't have to, the fact that the patent puts 90+% of the emphasis on other stuff looks like a strong indication that acting as a flat extra cache tier for the CPU is not intended to be its primary purpose even by a long shot.
Something to consider is that we tend to think in terms of best-case latency. However, when the SoC is under high load, the queues will fill up and the typical latency will actually be much greater. So, even if the latency for L4 is as high as @InvalidError estimates, it could still be a win vs. going all the way out to DRAM.
I don't think Intel would do that. The GPU has its own scratchpad memory, with relaxed coherency. For any memory visible to the CPU cores, it either needs to be treated as uncached, making it of limited usefulness to the CPU, or fully supporting x86's strong memory ordering. There's no real in-between.
Yes, and that's why DSPs and GPUs tend to have scratchpad memory. For general-purpose CPUs, the problem is that you'd have to flush your scratchpad as part of your thread context. Since paging in & out your scratchpad would get expensive, it's easier just to rely on caches.
If the scratch pad were treated as thread-private, then it would be fine. No different than registers, really.
I disagree. HBM should look like main memory, except the OS just needs to be aware that it's faster and therefore needs to actively migrate pages in and out of it.
That's different both in magnitude and kind. This supposed L4 cache will be 2-3 orders of magnitude smaller. I think the OS will be primarily tasked with managing HBM, but I concede that I have yet to see details of how it's meant to work. In contrast, I expect ADM will be hardware-managed.
It's not just a bandwidth problem, but also a matter of energy-efficiency. Data movement is energy-intensive.
I'd suggest you should be reading patents differently than design documents or whitepapers. Maybe the patent doesn't focus on it as a cache, because that would simply be indefensible.
With the number of patent lawsuits over "same old crap we did over postal service, phone or in-person but over e-mail, IP, Twitter, pigeons, etc." you cannot downplay the obvious stuff too much when you add alternate functionality unless the obvious function is of limited importance.
What would the alternate functionality be in the case of them using cache as cache?!
Patents have to be pedantically inclusive either explicitly or by generalization if you don't want to get screwed over by someone else patenting fundamentally the same thing with the obvious omissions put in.
Very true. ADM acting as a L4 cache will introduce too steep a penalty. Looks like it's destined for the tGPU primarily.
Actually, by nature, ADM L4 adds significant latency. This latency becomes a huge issue when cache misses exceeds cache hits. So, when the cache hits are low, the significantly increased DRAM access time will seriously degrade tCPU performance. ADM working as L4 seems very unlikely.
From the patent, it seems far more likely that the "L4$" will effectively act as a NUMA region partitioned between different uses by its drivers/OS instead of as conventional cache.
This was leaked a long while ago. And at that time, many didn't notice. It all makes sense now!
Read latency can be masked by prefetching and speculative loads.
I was thinking about this, but it would only work if you can invalidate the memory transaction should the contents of L4 be dirty. Remember that if it's working as a system-level cache, then it needs to be kept coherent.
This is exactly why I think they won't do it. With Meteor Lake being mobile-focused, power-savings are very important.
If the display is in power-saving mode, then I can understand powering down the GPU. Obviously, they're going to support powering down the CPU tile, with those 2x LPE cores in the SoC tile.
Seems L4 is ultra power efficient and doesn't need to be powered down. Instead, in the new low-power state, when the frame buffer doesn't need to be updated, the video memory can be buffered in the cache while idling and the tGPU can be powered-off as well.
Facebook
Twitter
Instagram