Discussion Asus PRIME Z690-P: DDR5 support is *bad* and its getting worse.

[palindrom]

Hi,
after quite some time with that board, all the problems and discussion with Asus, I decided to share a couple of thoughts I have on the matter.

Background:
I've built a new system. Last one was running since 2009 so about time.
Asus PRIME Z690-P
i7-12700KF
2x 2x16GB Kingston Fury Beast KF560C40BBK2
RTX3060
2x M.2 EVO980Pro
ROG Strix 850W.
I started with just 32GB kit, 2x16GB RAM and set it up in A2-B2 slots, XMP-I 6000MT/s.
It all worked flawlessly.
And then I decided to populate remaining 2 DIMM slots with identical set of memory.
--
It didn't even start. Having some doubt in my sanity, I started leafing through the documentation. But it was clearly there, 128GB DDR5 up to 6000MT/s in 4 modules.
So I did what I do at work, I began testing and obtained these results over 2 days:

Mainboard:
ASUS Prime Z690-P
P/N 90MB19Q0-M0EAY0
MFG 12/2021
BIOS version 2002

Modules type:
Kingston Fury Beast KF560C40BBK2-32 (2 sets, 16GBx4 total)
DDR5 6000MT/s CL40-40-40 1.35V@6000, XMP3

Modules serial numbers table:
A *593
B *592
C *390
D *391

All tests performed at 6000MT/s XMP I, because XMP II don't work.

RAM slots and memory modules configuration

No    A1    A2    B1    B2    Result

1    A    C    B    D    - FAIL 62 errors in 4 passes, from brief analysis errors were singular bit flips.
2    D    B    C    A    - FAIL 213 errors in 4 passes
3    A    D    B    C    - FAIL, test aborted
4    -    C    -    D    + PASS no errors (2 passes)
5    -    A    -    B    + PASS no errors (2 passes)
6    C    -    D    -    - FAIL did not boot, BIOS fallback to 4000MT/s
7    A    -    B    -    - FAIL did not boot, BIOS fallback to 4000MT/s
8    A    -    -    -    - FAIL did not boot, BIOS fallback to 4000MT/s
9    -    A    -    -    + PASS no errors (1 pass)
10    -    -    A    -    - FAIL did not boot, BIOS fallback to 4000MT/s
11    -    -    -    A    + PASS no errors (1 pass)
12    -    -    A    B    + PASS no errors (2 pass)
13    -    A    B    -    - FAIL did not boot, BIOS fallback to 4000MT/s
14    A    B    -    -    ? ???? did not boot, BIOS fallback to 3600MT/s, set to XMP 4000MT/s and rebooted successfully, then set to 6000MT/s on next reboot after XMP was turned on again. Tests PASS after that.
    
    Switched XMP off, speed set to 4800MT/s
15    A    D    B    C    + PASS no errors (4 passes)

The was fun!

I reached out to Asus for comments. My best guesses were bad timings after training routines or bad impedance control (not recoverable).
I got response that 4-module sets are not on a market yet so they're are not on compatibility lists. And 2x 2-module set is not the same.
I didn't find that response acceptable because MB is clearly not in line with specification. Modules are provided with all the maximal values of metrics on them so if used within stock settings, they're identical. I actually measured actual timings in nanoseconds just to verify.
What's even more annoying is that if MB and RAM manufacturers adhered to standards on both ends, we wouldn't have that lists in a first place.
So I got stuck with slower speed RAM for now and hoped for BIOS updates.
Well...

BIOS 2002 XMP off, speed set to 4800MT/s
15    A    D    B    C    + PASS no errors (4 passes)

BIOS 2014 XMP-I 5600MT/s
16    A    D    B    C    + PASS

BIOS 2212 XMP-I 5600MT/s
17    A    D    B    C    - FAIL (thousands of errors within 1 hour)

I did a fallback to version 2014.

Aftermath
I learned that well known mainstream PC hardware manufacturer pushes half-baked products to the market. A couple of years ago this wasn't that common. It looks like around DDR3 era, looking at basic signal metrics and observing eye diagrams at factories became optional.
If it didn't, we wouldn't have QVL.

Question
Do we have this kind of problems with all manufacturers, or was I just unlucky?
Not that I'm interested in continuation of my adventure with Asus. Not anymore. Not after I bought Asustor NAS and that MB.

 

[Eximo]

My understanding is that this is fairly common with trying to get 4 sticks working with Intel.

 

[palindrom]

Less problems with AMD?
If true, I'm getting a bit disappointed with my life choices.

 

[Roland Of Gilead]

Hey there,

Although they are two kits of the same ram, unless they are part of a quad pack, you are just not guaranteed to get them to work. Not a great choice. Even though the DIMMs may be the same in every respect, they are built with different silicon. They are NOT the same.

There are a couple of options:

1. Put all DIMMS in, and go to bios, and set everything to Auto for the mem. See if they all boot together at stock speed (2133/2400). If they work, then you may be able to get them to work by manually configuring them. Trying different speeds to until you get the desired speeds.

2. By a set of Quad packed DIMMs ( so you know they work together) and sell your current two sets to offset the difference.

3. Try a bios update, as this can bring mem compatibility, performance enhancements among other things.

Be sure to clear CMOS after the bios update.

 

[Roland Of Gilead]

Less problems with AMD?
If true, I'm getting a bit disappointed with my life choices.

No actually. Similarly AMD suffer with 4 slots occupied compared to 2. Typically current AMD CPU (Zen 3) support up to 3200mhz with 2 slots filled (it does often work with faster ram up to OC 4000mhz). With 4 slots filled AMD only say you can get a max of 2666mhz!!! WTF, right!!

It;s very similar for both Intel/AMD. Once you populate all slots, it immediately puts pressure on the IMC of the CPU, but also on the max supported speed of the mobo. Sometimes kicking up the SOC voltage can help here.

BTW Are you OC anything?

 

[Eximo]

Haven't heard much about 7000 series memory issues yet, but I did know about the reduced speed thing.

I had kind of hoped since they went second on DDR5 support that they would have learned a few lessons. Might be something they fix over time with AGESA updates like last time. Or maybe they've already baked improvements into the X3D chips, and subtle refreshes of the IO die over time will make things better.

Intel, they will probably just make DDR5 improvements on the next gen chips. (maybe the long term Raptor/Alder chips will also get some updates)

 

[palindrom]

No actually. Similarly AMD suffer with 4 slots occupied compared to 2. Typically current AMD CPU (Zen 3) support up to 3200mhz with 2 slots filled (it does often work with faster ram up to OC 4000mhz). With 4 slots filled AMD only say you can get a max of 2666mhz!!! WTF, right!!

It;s very similar for both Intel/AMD. Once you populate all slots, it immediately puts pressure on the IMC of the CPU, but also on the max supported speed of the mobo. Sometimes kicking up the SOC voltage can help here.

BTW Are you OC anything?

No, I'm running stock everything in my system. This is the first time I got such an instabilities with recommended configurations. It looks like manufacturers indeed hit the ceiling while struggling to take the market with false advertising.

BIOS update made things worse, i reverted to v2014.

But I have to disagree on memory kits here. Or at least point out what shouldn't be.
While it's critical to have all the actual parameters identical when pushing parameters beyond their limits, when staying at declared profile, external parameters must fall within declared limits. If I had 4 modules, each from different manufacturing date and different vendor, but all running declared 6000MT/s, 1.35v, 40-40-40-80, from the motherboard perspective, they have to be electrically identical. Same signals out, same logical levels, eye diagrams within spec, delays not greater than declared number of cycles, impedance matched. Unless we're betting on which stick will pull off 2ns less and just stay at the maximal values, they should work. If they don't, it would mean that hardware interface specification doesn't exist or manufacturers ignore it.
But even if memory is running dead on the edge of all declared parameters, my modules are close in serial numbers.

Actually... I could try to set up full speed but increase timings to test that hypothesis. I'll do it as soon as I'll replace CPU today or tomorrow.
Since I'm replacing i7-12700KF with i7-13700K, I'll check if a bit better memory controller will fare better. It shouldn't but let's see.

 

[Roland Of Gilead]

BIOS update made things worse, i reverted to v2014.

You're clearing CMOS each time, yeah?

Actually... I could try to set up full speed but increase timings to test that hypothesis. I'll do it as soon as I'll replace CPU today or tomorrow.
Since I'm replacing i7-12700KF with i7-13700K, I'll check if a bit better memory controller will fare better. It shouldn't but let's see.

Let us know how it goes.

 

[palindrom]

I'm back and not-so-happy.
Running 4x16GB at full speed but...

BIOS	Proc	Freq	Errors	Remarks
2014	i7-12700KF	6000	1098
2014	i7-12700KF	5600	PASS
2014	i7-13700K	6000	1	Running hot
2212	i7-13700K	6000	PASS	Running hot

Takeaways

So it looks like at the date of release Asus couldn't deliver on advertised memory speed due to CPU constrains and some problem with PCH. The latter one I know from Asus tech. Intel was too optimistic on declarations to say the least.

6000MT/s is right on the edge and I'm a bit worried about reliability.

13gen i7 eats 1/4 kilowatts of power and I'm now forced to replace my new Fortis5 dual fan tower radiator with water cooling.

I should have chosen AMD.

 

[DSzymborski]

This has always been the case with multiple DIMMs and will continue to be until such time that semiconductor fabrication is far, far more advanced than is possible for us with current or near-future technology. You're certainly free to pretend binned parts are truly identical, but that doesn't make it actually so. Semiconductor fabrication is not like manufacturing cars or bread or hats.

 

[palindrom]

There are no identical memory modules.
CL40 can be 37, 38 or 39.5 but never 41. Reported latencies must fall within maximum values in datasheet, so running memory at these values have to work. If it doesn't, that means modules are faulty or they didn't meet the spec.
Following cars analogy, if a goal is to have 4 of them simultaneously at the finish line, make all them speed limited to 20km/h. Then it doesn't really matter what their real parameters are.