maybe you can take it too far. If you have a 128 cores, how much do they even want or need to talk to each other?
Modern software (both kernel and userspace) assumes memory accesses are cache-coherent. This means a certain amount of traffic between the cores to support cache coherency, even when they're not actually exchanging data.
Maybe keeping "just" 16 or 32 of them in a package would cover 90% of any need, if there is any need?
For at least 5 years or so, AMD has supported partitioning EPYC CPUs into separate NUMA domains. I'm not sure, but I think it could be simply talking about de-interleaving memory accesses to an extent. Here's what the Zen 4 EPYC doc says about it:
"Beginning with AMD EPYC 7002 Series processors, non-uniform latency was reduced dramatically by locating memory controllers onto the I/O die. NUMA domains were flattened more by the move to 32 MB of L3 cache in the EPYC 7003 Series. In 4th Gen EPYC processors, optimizations to the Infinity Fabric interconnects reduced latency differences even further.
Using EPYC 9004 Series processors, for applications that need to squeeze the last one or two percent of latency out of memory references, creating an affinity between memory ranges and CPU dies (‘Zen 4’ or ‘Zen 4c) can improve performance. Figure 9 illustrates how this works. If you divide the I/O die into four quadrants for an ‘NPS=4’ configuration, you will see that six DIMMs feed into three memory controllers, which are closely connected via Infinity Fabric (GMI) to a set of up to three ‘Zen 4’ CPU dies, or up to 24 CPU cores.
Most applications don’t need to be concerned about using NUMA domains, and using the AMD EPYC processor as a single domain (NPS=1) gives excellent performance. The AMD EPYC 9004 Architecture Overview provides more details on NUMA configurations and tuning suggestions for specific applications"
Source: https://www.amd.com/system/files/documents/4th-gen-epyc-processor-architecture-white-paper.pdf
I'm pretty sure that's just a memory-partitioning trick and assumes the OS is careful not to migrate threads between NUMA domains (otherwise, performance penalties would result).
I think the main argument for packing so many cores in a single CPU is that it allows cloud operators to pack more cores per rack unit. Also, each physical machine incurs some management overhead for them. So, it works out to a cost-savings for them.
There are also some workloads that really want lots of cores and/or lots of memory capacity or bandwidth. If you're a cloud operator, you can charge a premium for these instances, that wouldn't be possible on smaller CPUs, and still fill out any remaining capacity on the machine with smaller VMs that have much more modest requirements.
With mobs of cores like that they are still hardwired to whichever L1, L2 caches they are using so are already somewhat constrained on memory and data fragmentation or duplication, so putting them all under a single cover is already misleading.
Misleading? What's misleading about having distributed caches? Go back as far as you like, even to the days before multi-core CPUs. Back then, each CPU had its own cache(s). There was never a point, in probably at least the last 3 decades, when a multi-core system didn't have such a cache architecture.
Facebook
Twitter
Instagram