News AVX-512 is a gamechanger on Intel Emerald Rapids CPU — 5th Gen Xeon runs twice as fast on average with slightly higher power consumption

[Admin]

A review details how much faster Intel's 5th Generation Xeon Emerald Rapids processors are with AVX-512; in some cases, performance is over ten times higher.

AVX-512 is a gamechanger on Intel Emerald Rapids CPU — 5th Gen Xeon runs twice as fast on average with slightly higher power consumption : Read more

 

[bit_user]

AVX-512 is a gamechanger on Intel Emerald Rapids CPU — 5th Gen Xeon runs twice as fast on average

Not typically. The GeoMean is skewed by the relatively large number of OpenVINO tests. That's a deep learning framework written by Intel and obviously quite optimized for its own hardware.

The median speedup for AVX-512, in those tests, is probably closer to 30%. That doesn't make for such an attention-grabbing headline, of course. And don't forget that the test suite was specifically tailored to include things which traditionally benefit from AVX-512. Across all computing workloads, the typical speedup would be in the (probably lower) single digits.

AVX-512 boosts performance up to 10X higher in some workloads.

Only in OpenVINO, and only for 2 of the test cases. Those tests were clearly designed to showcase specific AVX-512 (VNNI) instructions. If you didn't have those instructions, you wouldn't use that type of model, because other layer types perform better without VNNI.

It's one of the probable reasons Microsoft chose last-generation Sapphire Rapids chips over EPYC to pair with AMD's MI300X GPUs.

And not at all volume, cost, or time-to-market? Microsoft has their own AI accelerators, which could be another reason why they weren't interested in the GPU portion of MI300X.

 

[thestryker]

And not at all volume, cost, or time-to-market? Microsoft has their own AI accelerators, which could be another reason why they weren't interested in the GPU portion of MI300X.

MI300X is the OAM accelerator only, the MI300A is the APU version. Article author is just positing that this may be a reason why MS went SPR instead of Genoa (their Azure AI instances are SPR+MI300X).

 

[d0x360]

Not typically. The GeoMean is skewed by the relatively large number of OpenVINO tests. That's a deep learning framework written by Intel and obviously quite optimized for its own hardware.

The median speedup for AVX-512, in those tests, is probably closer to 30%. That doesn't make for such an attention-grabbing headline, of course. And don't forget that the test suite was specifically tailored to include things which traditionally benefit from AVX-512. Across all computing workloads, the typical speedup would be in the (probably lower) single digits.


Only in OpenVINO, and only for 2 of the test cases. Those tests were clearly designed to showcase specific AVX-512 (VNNI) instructions. If you didn't have those instructions, you wouldn't use that type of model, because other layer types perform better without VNNI.


And not at all volume, cost, or time-to-market? Microsoft has their own AI accelerators, which could be another reason why they weren't interested in the GPU portion of MI300X.

But bigger numbers in the headlines of a CPU for servers regarding a limited use instruction set brings in way more readers!