Question best methods for locking in low power draw on Ryzen 5950X for server usage with Asus Dark Hero?

[Firestone]

I want to configure my PC for lower heat output and lower power draw under full load while running as a home compute server.

I have the following setup;
- Ryzen 9 5950X
- Asus ROG Crosshair VIII Dark Hero (link https://www.asus.com/us/supportonly/rog crosshair viii dark hero/helpdesk_manual/ )
- 128GB (4x32GB) 3200MHz Corsair RAM
- Noctua NH-D15 with push/pull 120mm/140mm fans + 120mm exhaust fan; CPU hitting ~68C under sustained full load

Are there any suggestions on what settings to look at in order to implement this?

Some considerations;

- the Asus ROG BIOS are pretty confusing, there are tons of pages with options and settings with little, if any, description, so I honestly am not sure where to begin with custom configuration

- I already enabled Eco Mode for the CPU under the BIOS, and I had thought that the Turbo Boost was disabled, but the CPU is still boosting to 4.0GHz under full load, I was hoping to maybe lock that back down to the base clock of 3.5GHz (?) ; the Asus BIOS under Eco Mode mentions something about "65W" but its not clear to me that the CPU is actually running at a 65W power limit, or how much power its drawing in general right now

- I have not changed the stock voltage on the CPU from the default which I think is 1.44V ; from past experiences I know that reducing voltage can reduce power and heat more than reducing clock speed

- getting the XMP / DOCP 3200MHz memory profile to work stable-ly and pass Memtest86 was a bit of an ordeal so I am not sure if messing with the CPU voltages or settings will affect it? Am I going to have to re-run Memtest86 for this? It takes about 22hrs to complete

- I am running Ubuntu 22.04 so my ability to tweak any kind of power settings from within the OS is pretty much non-existent (there is not Ryzen Master or MSI Afterburner for Linux)

- I am paying for electricity and the PC sits next to my desk so I have a strong incentive to keep the power and thermals low here LOL

Let me know what you all think, its been a very long time since I messed with under/over clocking and undervolting heavily, the Ryzen CPU's seem to handle their own power management exceptionally well when left to their own devices, but in this case of having a sustained 100% load it feels like I am fighting the gamer-centric stock behavior that wants to keep speeds high at the cost of power and heat. So ideally I would only have to make as few adjustments as possible and then let Ryzen sort itself out.

 

[punkncat]

I have zero hands on experience with the 5000 series of Ryzen CPU, so what follows is some pontification and rambling about my past experience with Ryzen, in general (and also Windows centric)

As you are probably already aware, one of the power profiles that is "Ryzen optimized" involves the CPU being kept at a higher clock even when idling in order to make them more responsive. Even if you set the power plan for a Ryzen at "balanced" the clocks will not drop to the same idle rate that (for instance) an Intel CPU will. In spite of this, Ryzen seem to using much less power than it's comparable Intel counterparts.

With that said, I think there is something to be considered for just using the CPU at its design parameters. I don't know what your use case for this server is, but IME it doesn't take a CPU nearly that powerful just to handle some file sharing and such. This would be something you have to consider in light of its use case.

The motherboard power delivery (VRM) is supplying voltage to the CPU and to the memory. XMP/DOCP is an overclock and changing the power profile parameters of one (CPU or RAM) can change the behavior of the other. A direct example of this would be such as the differences in (RAM) clock between using 2 sticks and 4, with the latter sometimes not being able to be set as high as when only using 2.

 

[Firestone]

I don't know what your use case for this server is, but IME it doesn't take a CPU nearly that powerful just to handle some file sharing and such. This would be something you have to consider in light of its use case.

right, I already have a separate low power system doing file sharing, this system would be basically running BOINC at 100% load 24/7 in the background while "idle", and then running whatever computational or AI / ML loads I feel like as needed on top of that. So the system will be pegged at 100% usage pretty much constantly on the CPU, and light loads on the GPU too.

Before setting the CPU's Eco Mode, it was clocking up around 4.4GHz under loads, afterwards it came down to about 4GHz. Since I dont necessarily care about speed I am fine with bringing that lower if it helps power and thermals. Right now the system is pulling something like 300-400W at the wall (kinda hard to tell exactly since the Kill-a-Watt is on the same circuit as my other light home server systems), the PC is pretty much silent but the small amount of heat buildup is noticeable over time.

Definitely dont want to upset the XMP/DOCP RAM configuration on this one since it was such a PIA to get working.

 

[punkncat]

Again, rather thinking out loud than having a helpful answer...does this type of server require a powerful GPU as well? I understand a lot of the AI workloads benefit from a powerful graphics card. If it doesn't, that would be a great place to make a cut in power use.

 

[Firestone]

current GPU is the RTX A4500, got it running as well but honestly its power impact has been pretty minimal, most of the workloads its running on BOINC are barely using 50W whereas it will be under full load when I switch over to AI stuff

also worth noting that from experience, changing GPU clock speeds and voltages in Ubuntu / Linux is a nightmare that I have never been able to figure out how to do successfully myself, there is no easy interface to it like on Windows, all my attempts have just ended up crashing the system one way or another.

 

[hotaru.hino]

If you don't care about clock speed or absolute performance of the CPU, I would say lock the frequency to the base frequency and start with the CPU voltage at 1V. If it boots and it can run a multithreaded workload, lower the voltage gradually until either you hit the floor or the system crashes.

Then if you want, you can tweak the other voltages like the VTT_SOC, VDDG, and VDDP. Though in recent testing when I did this, I found this only takes out 3-4W from the wall.

 

[wyliec2]

If I'm understanding your use case correctly, then possibly PBO is of little/no value to you??

I run a 5950X on Crosshair VIII Formula primarily for encoding with Handbrake. It's not uncommon for me to run 7-10 days at constant 90%+ CPU load.

I have settled on an all-core OC - my go-to value is 4200 with a very small positive offset voltage. Over several days constant use, my max temp is 68C and average 66C; max CPU core voltage of 1.184 and average 1.14v.

Given your desires, you could set an all-core value of 3500 and experiment with undervolting with small negative offset values.

I find the 5950X to be easy to set up for very efficient operation with low heat and power use. I also use a 10700K and it's a hog in terms of power and heat....

 

[Firestone]

well the other half of the issue here, is how confusing the BIOS for Asus ROG motherboards can be.

Seems like there's at least two separate pages in BIOS for these settings, both with slightly different settings and options, one of them under the "Extreme Tweaker" page (also called "AI Tweaker" on some boards it seems), then another under "Advanced > AMD Overclocking"

Both pages seem to have AMD Precision Boost Overdrive (PBO) settings

Confusingly, I had Eco Mode enable previously, but after fiddling with other settings and disabling PBO, the options for Eco Mode vanished. It was not until I finally made sure that PBO was set to Auto (Extreme Tweaker > Precision Boost Overdrive), that I eventually (after some reboots) got Eco Mode settings to appear again under Advanced > AMD Overclocking > Eco Mode > Eco Mode (65W).

Now after enabling that setting, it seems to have automatically adjusted other settings;
- Precision Boost Overdrive [Auto] -> [Advanced]
- PBO Limits [Auto] -> [Manual]
- PPT Limit [W] [0] -> [87]
- TDC Limit [A] [0] -> [60]
- EDC Limit [A] [0] -> [90]

After this was applied, now the system booted in Ubuntu and when running under full load the CPU appears to be staying at 2.6GHz or less which comes to about 57C with my current setup.

This is not actually what I wanted, its quite a bit lower, however, its good enough for now I guess, since the decrease from est. 70C has made a pretty huge difference in heat output. So I guess I will take it. Somehow though previously when I had this same Eco Mode enabled, it was not this restrictive. So something in the settings might be misconfigured still. Not sure. Its honestly pretty difficult to deal with this stuff in the Asus ROG BIOS, there's so many similarly named settings scattered in multiple places, and adjusting some appears to change others, sometimes disabling options completely in other pages, its quite a headache.

 

[Firestone]

If I'm understanding your use case correctly, then possibly PBO is of little/no value to you??

I run a 5950X on Crosshair VIII Formula primarily for encoding with Handbrake. It's not uncommon for me to run 7-10 days at constant 90%+ CPU load.

I have settled on an all-core OC - my go-to value is 4200 with a very small positive offset voltage. Over several days constant use, my max temp is 68C and average 66C; max CPU core voltage of 1.184 and average 1.14v.

Given your desires, you could set an all-core value of 3500 and experiment with undervolting with small negative offset values.

I find the 5950X to be easy to set up for very efficient operation with low heat and power use. I also use a 10700K and it's a hog in terms of power and heat....

one of the really confusing parts of this is that technically, no, I do not want PBO

however, in the BIOS, the default state of PBO is "Auto"; I had at some point changed this to "Disabled"

it seems that in Disabling PBO, I also disabled Eco Mode, since counter-intuitively PBO must be set to Auto for Eco Mode to be an option in the motherboard BIOS.

 

[Firestone]

Also, for what its worth, I tried locking the CPU frequency at 3.5GHz, but while using the stock 1.45V settings, and this had almost no effect on thermals, it stayed in the 67-70C range (depending on ambient temp), the same as when set to 4.0-4.2GHz. Was not able to check the power draw at these settings though.

 

[wyliec2]

Since my initial reply, I've done more tweaking with Offset voltage - setting the offset value to +0.06250 and keeping the all-core OC of 4200. I actually was able to set it to a negative offset of -0.00625 and run stable at 4200.

After running my trial 4K Handbrake encode for an hour with average CPU utilization of 93%, I had a max temp of 57C and average of 55.9C. These values have held consistent after several days of 24/7 encodes.

This configuration delivers a Cinebench R23 multi-thread score of 27867.

For clarity - my settings are PBO disabled and everything at Auto or default values. The only changes are setting the OC to 4200 (42.00), Voltage to Offset mode, negative (-) and entering .00625.

 

[Firestone]

Thanks for that update, those are some interesting results.

I wanted to give some more updates into these details, since its becoming apparent to me now that there's more going on here than I first thought

It was not until I finally made sure that PBO was set to Auto (Extreme Tweaker > Precision Boost Overdrive), that I eventually (after some reboots) got Eco Mode settings to appear again under Advanced > AMD Overclocking > Eco Mode > Eco Mode (65W).

Now after enabling that setting, it seems to have automatically adjusted other settings;
- Precision Boost Overdrive [Auto] -> [Advanced]
- PBO Limits [Auto] -> [Manual]
- PPT Limit [W] [0] -> [87]
- TDC Limit [A] [0] -> [60]
- EDC Limit [A] [0] -> [90]

After this was applied, now the system booted in Ubuntu and when running under full load the CPU appears to be staying at 2.6GHz or less which comes to about 57C with my current setup.

So for context, my main purpose for running continuously 24/7 at "low" power and heat configurations is to allow for running BOINC science compute on my PC while I am not using this. I am running this primarily under Linux (Ubuntu 22.04). Notably, BOINC typically configures itself to run up to 100% CPU utilization, however, it seems to give its task processes a `nice` value of 19, which sets the process priority to the second-lowest possible value (`nice` values range from 20 to -20, with higher being "nicer" and thus lower priority, lower values <0 typically being high priority system and kernel tasks, etc.).

So running BOINC now with these settings, my CPU is at 100% utilization but with level 19 ultra low priority processes. During this time, `btop` is reporting that my CPU clock is running in the ballpark of 2.7-2.8GHz, and as @ wyliec2 described, the thermals are in the range of 55-57C at this time.

example pic here

https://imgur.com/a/9TisAbV

View: https://imgur.com/a/9TisAbV


For context, the default minimum clock speed I usually see on this CPU (in `btop`) with the Eco Mode enabled, in Ubuntu 22.04, at full idle, is reported around 2.2GHz; the box-spec base clock for the CPU is listed as 3.4GHz online, and when I am putting the CPU under full load, with my current PBO settings I usually see it clock up to 4.0-4.2GHz. So, to have the CPU at full load but still remain at 2.8GHz is a pretty interesting situation, and its actually pretty close to what I wanted in the first place.

However the situation changes if I return to using the CPU; under these exact same settings, if I turn off BOINC and go back to normal usages (e.g. 100% CPU processes at the default `nice` level 0) then the CPU will go back to the old behavior of clocking up to 4.0GHz+, with heat output to match (68-70C).

All of this testing is in Ubuntu; I have not yet figured out a good method for determining the detailed CPU power usages in Linux yet, and I am not 100% clear yet if this behavior is specific to Ubuntu / Linux. Because, when I switch back to Windows, the Task Manager does not show me the CPU downclocking to 2.2GHz on idle, instead it appears to drop only to 3.5GHz, and then back up to 4.0-4.2GHz under full load. I suspect I might need to re-install the Ryzen Master app on Windows to really get the full picture of what is going on.


Regardless though, since this PC will mostly be sitting on Linux while idle running BOINC, I think I find the current situation of clocking down to ~2.8GHz for low-priority BOINC tasks ideal, and the CPU still responds well when I switch back to normal high-usage tasks. And, all of this is at the stock voltage of 1.45V as well. If anyone has suggestions for how to get sensible CPU power metrics (Watts) readings in Linux, that would help to round out this picture, I had attempted some things I found on Stack Overflow but they did not really work the way I needed.

 

[Firestone]

Also, I found this thread which sheds some light on what is going on here, I think

Understanding the sizing and limits of PPT TDC and EDC

www.overclock.net

Package Power Tracking (“PPT”): The PPT threshold is the allowed socket power consumption permitted across the voltage rails supplying the socket. Applications with high thread counts, and/or “heavy” threads, can encounter PPT limits that can be alleviated with a raised PPT limit.

Default for Socket AM4 is at least 142W on motherboards rated for 105W TDP processors.
Default for Socket AM4 is at least 88W on motherboards rated for 65W TDP processors.

Thermal Design Current (“TDC”): The maximum current (amps) that can be delivered by a specific motherboard’s voltage regulator configuration in thermally-constrained scenarios.

Default for socket AM4 is at least 95A on motherboards rated for 105W TDP processors.
Default for socket AM4 is at least 60A on motherboards rated for 65W TDP processors.

Electrical Design Current (“EDC”): The maximum current (amps) that can be delivered by a specific motherboard’s voltage regulator configuration in a peak (“spike”) condition for a short period of time.

Default for socket AM4 is 140A on motherboards rated for 105W TDP processors.
Default for socket AM4 is 90A on motherboards rated for 65W TDP processors.

Notably in the Asus ROG BIOS, when you select "Eco Mode", its actually listed as "Eco Mode (65W)". Based on the description here, and based on the automatically set values for PPT / EDC / TDC when you select "Eco Mode" on the 5950X, it appears that whats actually happening is that its apply a profile that sets the CPU's power metrics as if its a 65W TDP chip, instead of a 105W TDP. Pretty interesting. So I am guessing that under this configuration, the Power Limit for the CPU is closer to 88W.