AMD Unveils EPYC Server Processor Models And Pricing

[Guest]

AMD officially unveiled the breadth of its new EPYC server processor line, as well as pricing details, at its Tech Day in Austin, Texas.

AMD Unveils EPYC Server Processor Models And Pricing : Read more

 

[shang005]

Picture links broken. @4:03 EST

 

[ClusT3R]

Just Amazing they double every single performance, lets wait for some benchmarks but it looks very promise on paper.

 

[Blas]

Right below the first slide that details the cores/threads, base clocks and boost frequencies, it reads:
"EPYC's base processor frequencies range from 2.1GHz to 2.2GHz, which is a relatively small frequency range compared to Intel's lineup. We see a slightly higher delta between the 2.9-3.2GHz maximum boost frequencies"

But the slide shows base clocks ranging from 2.0 to 2.4 GHz, and boost clocks ranging from 2.7 to 3.2 GHz.

 

[PaulAlcorn]

Good eye, fixed!

 

[bit_user]

Nice article, Paul (and your Intel i9-7900X, as well... which I'm still reading)!

Good eye, fixed!

I think this:

VMs that are larger than four cores will spread across multiple CCX, thus regularly encountering the increased latency of the Infinity Fabric. AMD disclosed bandwidth specifications for the various connections, including the CCX-to-CCX and socket-to-socket interfaces. The design includes three connections from each CCX to neighboring CCX, thus providing a direct one-hop connection between all four of the core complexes.

should be:

VMs that are larger than eight cores will spread across multiple dies, thus regularly encountering the increased latency of the Infinity Fabric. AMD disclosed bandwidth specifications for the various connections, including the die-to-die and socket-to-socket interfaces. The design includes three connections from each die to neighboring die, thus providing a direct one-hop connection between all four of the dies.

My understanding is that communication between the two CCX's on the same die traverses a crossbar that's not considered part of the Infinity Fabric.

 

[bit_user]

Intel has a COD (Cluster on Die) feature it employs in high-core count Xeons to bifurcate the processor's disparate ring buses into two separate NUMA domains. It's a novel approach that reduces latency and improves performance, and a similar technology would significantly benefit AMD.

It seems like it should be easy for AMD to enable each die to have its own address space and run independently of the others. You could certainly do this with a VM, but there'd have to be some hardware support to gain the full benefits from it.

 

[bit_user]

Looks like a lot of good stuff in there.

The 128-bit FPU sticks out as one of the biggest chinks in their armor. I take their point about using GPUs to fill the gap, but this comes across weak by comparison with Intel's inclusion of some AVX-512 instructions in Skylake-X.

If they' could've at least matched Broadwell's 3-cycle FMUL latency, that would've helped.

 

[PaulAlcorn]

Nice article, Paul (and your Intel i9-7900X, as well... which I'm still reading)!

Good eye, fixed!

I think this:

VMs that are larger than four cores will spread across multiple CCX, thus regularly encountering the increased latency of the Infinity Fabric. AMD disclosed bandwidth specifications for the various connections, including the CCX-to-CCX and socket-to-socket interfaces. The design includes three connections from each CCX to neighboring CCX, thus providing a direct one-hop connection between all four of the core complexes.

should be:

VMs that are larger than eight cores will spread across multiple dies, thus regularly encountering the increased latency of the Infinity Fabric. AMD disclosed bandwidth specifications for the various connections, including the die-to-die and socket-to-socket interfaces. The design includes three connections from each die to neighboring die, thus providing a direct one-hop connection between all four of the dies.

My understanding is that communication between the two CCX's on the same die traverses a crossbar that's not considered part of the Infinity Fabric.

VMs that occupy more than four physical cores will spread across multiple CCX. The CCX are connected by the Infinity Fabric, which is a crossbar. AMD has indicated that the CCX-to-CCX connection is via Infinity Fabric. In fact, they claim the Infinity Fabric is end-to-end, even allowing next-gen protocols, such as CCIX and Gen-Z, to run right down to the CCX's. I did make a mistake on the last sentence, it is 'die,' thanks for catching that. It's been a long week

 

[Puiucs]

Looks like a lot of good stuff in there.

The 128-bit FPU sticks out as one of the biggest chinks in their armor. I take their point about using GPUs to fill the gap, but this comes across weak by comparison with Intel's inclusion of some AVX-512 instructions in Skylake-X.

If they' could've at least matched Broadwell's 3-cycle FMUL latency, that would've helped.

The reality is that GPUs are the ones that will provide that kind of raw power for servers. Not having AVX-512 is not that important (at least not yet). Server applications don't generally employ bleeding edge technology. AVX2 support should be more than enough for the majority of cases.

At the moment, its price or performance that will keep AMD from being used in servers, but the fact that it's considered bleeding edge technology. It will have to prove that it's a stable platform with no bugs or other issues in the long run. Server admins should not have to worry about bios and driver updates.

 

[bit_user]

I did make a mistake on the last sentence, it is 'die,' thanks for catching that. It's been a long week

Then it would be in the last two sentences.

 

[jimmysmitty]

Looks like a lot of good stuff in there.

The 128-bit FPU sticks out as one of the biggest chinks in their armor. I take their point about using GPUs to fill the gap, but this comes across weak by comparison with Intel's inclusion of some AVX-512 instructions in Skylake-X.

If they' could've at least matched Broadwell's 3-cycle FMUL latency, that would've helped.

The reality is that GPUs are the ones that will provide that kind of raw power for servers. Not having AVX-512 is not that important (at least not yet). Server applications don't generally employ bleeding edge technology. AVX2 support should be more than enough for the majority of cases.

At the moment, its price or performance that will keep AMD from being used in servers, but the fact that it's considered bleeding edge technology. It will have to prove that it's a stable platform with no bugs or other issues in the long run. Server admins should not have to worry about bios and driver updates.

Yes and no. In a data center they will still utilize a CPU heavily so anything that can help its tasks are welcome, otherwise all of AMDs slides up above are just as pointless as AVX-512 since the GPUs will be doing all the work.

 

[bit_user]

The reality is that GPUs are the ones that will provide that kind of raw power for servers. ...

Yes and no. In a data center they will still utilize a CPU heavily so anything that can help its tasks are welcome, otherwise all of AMDs slides up above are just as pointless as AVX-512 since the GPUs will be doing all the work.

Actually, there are plenty of GPUs in datacenters, and this trend will certainly continue. That and NVMe are two of the main reasons EPYC has so many PCIe lanes.

My concern is that there are a decent number of places where AVX is useful for small computations that are too fine-grained to offload onto a GPU, or where AVX-friendly computation is too closely interspersed with the sort of dense control-flow that GPUs hate.

Remember: SSE is 128-bit. That's all a Ryzen/Epyc core can issue in one clock. AVX needs two clock cycles. So, if you have an AVX multiply, Ryzen's 4-cycle latency and 128-bit pipeline (or dual-64 bit, I guess) means it takes 5 cycles to get your answer vs. 3 for Broadwell. And that's not even speaking of AVX-512.

 

[kyotokid]

...so my thinking is how would this fare for a CG workstation. Not all render engines support GPU rendering. 64 processor threads and say 256 GB of 8 channel memory is a healthy amount of horsepower. Almost don't need a render farm.

 

[jimmysmitty]

The reality is that GPUs are the ones that will provide that kind of raw power for servers. ...

Yes and no. In a data center they will still utilize a CPU heavily so anything that can help its tasks are welcome, otherwise all of AMDs slides up above are just as pointless as AVX-512 since the GPUs will be doing all the work.

Actually, there are plenty of GPUs in datacenters, and this trend will certainly continue. That and NVMe are two of the main reasons EPYC has so many PCIe lanes.

My concern is that there are a decent number of places where AVX is useful for small computations that are too fine-grained to offload onto a GPU, or where AVX-friendly computation is too closely interspersed with the sort of dense control-flow that GPUs hate.

Remember: SSE is 128-bit. That's all a Ryzen/Epyc core can issue in one clock. AVX needs two clock cycles. So, if you have an AVX multiply, Ryzen's 4-cycle latency and 128-bit pipeline (or dual-64 bit, I guess) means it takes 5 cycles to get your answer vs. 3 for Broadwell. And that's not even speaking of AVX-512.

My only point is that the CPU will serve a purpose otherwise Intel nor AMD would pursue better performance for them. I know that GPUs are becoming power horses, well for Intel Xeon Phi, in HPC because they can process a lot at once but CPUs still have a purpose.

 

[msroadkill612]

Given 5 year dev cycles, my guess is lane draining nvme took intel AND amd by surprise.

"That and NVMe are two of the main reasons EPYC has so many PCIe lanes."

 

[jimmysmitty]

Given 5 year dev cycles, my guess is lane draining nvme took intel AND amd by surprise.

"That and NVMe are two of the main reasons EPYC has so many PCIe lanes."

I doubt NVMe took Intel by surprise considering that they were one of the main companies behind it:

http://www.bswd.com/FMS09/FMS09-T2A-Huffman.pdf

That PDF is from 2009 and from the original group that specified the NVMe spec, NVMHCI.

That led to the creating of NVM Express Organization which was more companies coming in to set the standard since they will be using it, similar things happen for USB and DRAM.

So again I doubt Intel was taken by surprise. AMD is possible though since I can't find any information of them being involved. They had to know something was coming. Everyone knew AHCI was not the best possible for SSDs and that a new standard would be coming just for SSDs. AHCI was mainly developed with SATA HDDs in mind as IDE was too slow to handle the better speeds.

 

[fr33will]

Picture links broken. @4:03 EST

Same issue, but I noticed that it's privacy badger that is blocking the images.

 

[msroadkill612]

Given 5 year dev cycles, my guess is lane draining nvme took intel AND amd by surprise.

"That and NVMe are two of the main reasons EPYC has so many PCIe lanes."

I doubt NVMe took Intel by surprise considering that they were one of the main companies behind it:

http://www.bswd.com/FMS09/FMS09-T2A-Huffman.pdf

That PDF is from 2009 and from the original group that specified the NVMe spec, NVMHCI.

That led to the creating of NVM Express Organization which was more companies coming in to set the standard since they will be using it, similar things happen for USB and DRAM.

So again I doubt Intel was taken by surprise. AMD is possible though since I can't find any information of them being involved. They had to know something was coming. Everyone knew AHCI was not the best possible for SSDs and that a new standard would be coming just for SSDs. AHCI was mainly developed with SATA HDDs in mind as IDE was too slow to handle the better speeds.

Odd then there is currently ~NO affordable current desktop mobo with a lazy 8 pcie3 lanes free for a mere pair of nvme drives, assuming a 16 lane gpu in the system? Raid is out of the question w/o an expensive card & using only 8 lanes for gpu.

Some select intels are 28 pcie3 lane, some are 24, and some current models, just 16 lanes.

Ryzen is 24 usable on x370 chipset and 20 on x350.

Not prescient where i come from. Its missing the bleeding obvious. They need 4 lanes each!

"We knew how cool nvme would be, so we planned so u can only have one device on our systems"
Right

AMD happened to be targeting the server market where lanes matter for other reasons, and fabric made it easy for them to be generous in TR & Epyc. I fault them for being niggardly with AM4/ryzen.

I think many will opt for TR for the lanes, rather than the extra cores.

ps, i see i have contradicted myself ""That and NVMe are two of the main reasons EPYC has so many PCIe lanes.""

I retract that. More likely it was a fluke (as above) that serendipitously dovetails with these & other HB devices that are all the rage.

Its only IMO of course.

 

[jimmysmitty]

Given 5 year dev cycles, my guess is lane draining nvme took intel AND amd by surprise.

"That and NVMe are two of the main reasons EPYC has so many PCIe lanes."

I doubt NVMe took Intel by surprise considering that they were one of the main companies behind it:

http://www.bswd.com/FMS09/FMS09-T2A-Huffman.pdf

That PDF is from 2009 and from the original group that specified the NVMe spec, NVMHCI.

That led to the creating of NVM Express Organization which was more companies coming in to set the standard since they will be using it, similar things happen for USB and DRAM.

So again I doubt Intel was taken by surprise. AMD is possible though since I can't find any information of them being involved. They had to know something was coming. Everyone knew AHCI was not the best possible for SSDs and that a new standard would be coming just for SSDs. AHCI was mainly developed with SATA HDDs in mind as IDE was too slow to handle the better speeds.

Odd then there is currently ~NO affordable current desktop mobo with a lazy 8 pcie3 lanes free for a mere pair of nvme drives, assuming a 16 lane gpu in the system? Raid is out of the question w/o an expensive card & using only 8 lanes for gpu.

Some select intels are 28 pcie3 lane, some are 24, and some current models, just 16 lanes.

Ryzen is 24 usable on x370 chipset and 20 on x350.

Not prescient where i come from. Its missing the bleeding obvious. They need 4 lanes each!

"We knew how cool nvme would be, so we planned so u can only have one device on our systems"
Right

AMD happened to be targeting the server market where lanes matter for other reasons, and fabric made it easy for them to be generous in TR & Epyc. I fault them for being niggardly with AM4/ryzen.

I think many will opt for TR for the lanes, rather than the extra cores.

ps, i see i have contradicted myself ""That and NVMe are two of the main reasons EPYC has so many PCIe lanes.""

I retract that. More likely it was a fluke (as above) that serendipitously dovetails with these & other HB devices that are all the rage.

Its only IMO of course.

NVMe is still new and more of an "elite" thing rather than mainstream thing. I expect that to change within 2 years but we wont see true multi x4 PCIe M.2 slots in anything but the high end to start.

Most server still run RAID cards using SAS. However newer servers are blade servers that have a small flash (SSD) cache and they connect vi a 10Gbe connection to a SAN that holds all the data. The servers are literally just used for CPU cores and memory for multiple VMs. The SAN may move to flash based storage but it is still very expensive per GB to consider for the majority of companies.

Don't worry. It takes time but it will move on.