[SOLVED] Server PSU for multiple mobos

Page 2 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.
Oct 23, 2020
25
0
30
Hi
I want to build a 4-mobo custom box with a new Ryzen 9 5950x when available. The idea is to have one server PSU. Something like 80Plus Platinum 2000W+. Server PSU has only a 12V line and at the same time, it is the only voltage that has the most consumption nowadays. There are two ways I think to power it all today:
  1. normal low-power ATX PSU with 24pin y splitters + server PSU for powering CPU 4+4 lines and GPUs on all mobos
  2. Server PSU powering GPUs and CPU 4+4 lines and four Pico PSUs powering mobos.
My understanding that minimal 120W pico PSUs should be enough having CPU powered by server PSU.
Questions:
  1. Will that work or there are any obvious problems?
  2. Should I connect 12v GND with the GND of Picos or ATX PSU? So that +12V has a common GND.

P.S. Please don't post off-topic questions regarding the use case or why don't I just buy 4 ATX PSUs
 
Last edited:
Solution
WDYM primary? they are physically connected. They are the same 12V rail in your computer. Just wired through different paths. And the current will flow through whatever path has lower resistance. Meaning that there is no primary and secondary path.
Now if we have Pico powering ATX24pin and Server PSU powering 4+4 CPU EPS connector what you say can be either true or false depending on where the 12V will be higher. And technically it can be higher on the 24pin side and in that case Pico might end up overloaded. Though now I see there are pico PSUs out there that just pass through the 12V line without any DC-DC on it. I suspect they need to invert that 12V line for -12V pin though

Thats exactly what I mean, the power is coming...
Oct 23, 2020
25
0
30
No 2000w Server PSU from any reputable manufacturer that I have seen has a 24 pin on it, because server motherboards do the breakdown you see in that connector, so unless you find a way to not feed 166amps into the motherboard connectors which can't handle that, what can happen? You burn the connector, or you fry the chipset, or worst case you fry the CPU (unlikely though). Theres a reason the 24 pin is broken down like that, because different parts of the connector go to different parts of the board, and you don't want to exceed the power capacity of the wiring.

Now, Is there a way to do this? Well yes there sure is, but its complicated, and from your line of questioning beyond your level of electronics expertise. You're not doing it with adapters, you are going to need to bust our the soldiering iron, some capacitors, resistors, etc. Its one of those if you have to ask you probably shouldn't be doing it type of deals.



If you're paying $50 for them they are used, there is no way you're buying new ones for that price unless they are stolen, or cheap junk. They are between $300 and $400.

If you're finding ones with 24 pin connectors on them, then they aren't quality units. I don't care about the Platinum rating, that is efficiency only and has no bearing on quality.
I'm sorry if it sounded like I want to offend you because I wasn't.
We were not talking about providing 24pin ATX directly from the server PSU. We're talking about providing 12V to Pico PSU -> ATX 24pin. OR one ATX psu -> y-splitter-> four ATX 24pin. And adding to that direct 12V from the server PSU to 4+4 CPU connector. I'm not really sure if it is safe. I know that CPU is similar to GPU in regard to the voltage they use. They both use 0.4-1.5V range. And both have VRM circuits to convert from 12V. Technically, if GPUs work just fine from the separate PSU CPU should also be happy. In that case the power consumption from the Pico PSU should be really low and I can run the mobo from like 120W Pico PSU or four mobos from 400W ATX PSU with Y-splitters.
You see the point?
Now regarding the used PSU. Yes, they are used. Doesn't necessarily mean they are bad. Just replaced because the system they were running in cost >50k USD new and it should be replaced because of the critical load these servers serve.
 

Rogue Leader

It's a trap!
Moderator
I'm sorry if it sounded like I want to offend you because I wasn't.
We were not talking about providing 24pin ATX directly from the server PSU. We're talking about providing 12V to Pico PSU -> ATX 24pin. OR one ATX psu -> y-splitter-> four ATX 24pin. And adding to that direct 12V from the server PSU to 4+4 CPU connector. I'm not really sure if it is safe. I know that CPU is similar to GPU in regard to the voltage they use. They both use 0.4-1.5V range. And both have VRM circuits to convert from 12V. Technically, if GPUs work just fine from the separate PSU CPU should also be happy. In that case the power consumption from the Pico PSU should be really low and I can run the mobo from like 120W Pico PSU or four mobos from 400W ATX PSU with Y-splitters.
You see the point?
Now regarding the used PSU. Yes, they are used. Doesn't necessarily mean they are bad. Just replaced because the system they were running in cost >50k USD new and it should be replaced because of the critical load these servers serve.

I disagree on running what will be expensive hardware on a used PSU. Server rooms are harsh environments, and yes I understand service life and all that, but you don't know who is selling these or where they came from or how hard they were run.

I understand what you're saying about how you want to do this, however don't forget that the 4+4 is supplemental to the CPU drawing power from the motherboard. The CPU doesn't just say "eh the board PSU is a bit weak let me draw from the socket". You're cutting it SUPER SUPER CLOSE trying to run the motherboards off a 120w Pico or a 400w ATX. You're working under the assumption that it will always be running under a certain wattage and never have a spike.

Mining loads are consistent, there are no spikes, its a very smooth power draw. I'm actually less concerned about your desire to split up the 2000w server PSU, and more about how you plan to normally power the motherboards, and have these things work together.

No, none of this is safe, and it really sounds like an overcomplicated way to cheap out on a PSU. You could just run 4 higher end Seasonic units with 10 year warranties, and not do all of this ridiculousness. Yes it will cost more, but if one of them fails and takes out your VERY EXPENSIVE hardware (it won't) at least there is a warranty to back it up.
 
I'll pipe in here and say this is a bad, bad, bad idea.

Buying 4 5950X + 4 Mobo + 4x however much RAM yu need + 4 GPUs, + 4 SSDs and/or HDDs is not going to be worth it compared to buying a 64-core Threadripper, even if the 3990X is a little slower than a 16 core 5950X.

Hell, it'll be more expensive than just buying a 64-core EPYC server.

and you want to run this Frankenstein monstrosity on a $50 used power supply? (read: Worn the F**K out. No one swaps out a server power supply unless they absolutely HAVE to, meaning used ones are about to die any hour now.

WHY in gods name are you trying to save two or three hundred dollars on what is already a $7000 piece of madness? 4 decent quality 650W 80+ Bronze PSUs from Antec, Silverstone, Seasonic, Be Quiet! or Corsair will be not much more than the pico+Server PSUs you are looking for in the first place.

You now have four Tom's Hardware Moderators telling you its a bad idea. Maybe it's actually a bad idea?
 
  • Like
Reactions: melco
Oct 23, 2020
25
0
30
You're working under the assumption that it will always be running under a certain wattage and never have a spike.
Not really. CPU VRM has three 12V lines that are all the same 12V: MB internal, EPS 4pin + EPS 4pin. When the power draw grows the electricity will come from:
a) higher voltage source
b) related to (a): a lower resistance
Power will be sourced from 24pin atx only in one case: 12V line is significantly higher than 12V line of EPS connector (server PSU)
So electrical design is not a gamble. VRM just have 3 connections. And EPS lines are needed because they are thicker and close to VRMs. Nothing more :) Correct me if I'm wrong.
No, none of this is safe, and it really sounds like an overcomplicated way to cheap out on a PSU. You could just run 4 higher end Seasonic units with 10 year warranties, and not do all of this ridiculousness. Yes it will cost more, but if one of them fails and takes out your VERY EXPENSIVE hardware (it won't) at least there is a warranty to back it up.
This is basically true and I agree with that. It's always a safer way. I wanted to see if my assumptions about the wiring all that with one PSU are correct or you can find any flaw besides the nonsense of trying to save another 300$ on such a build
 
Oct 23, 2020
25
0
30
I'll pipe in here and say this is a bad, bad, bad idea.

Buying 4 5950X + 4 Mobo + 4x however much RAM yu need + 4 GPUs, + 4 SSDs and/or HDDs is not going to be worth it compared to buying a 64-core Threadripper, even if the 3990X is a little slower than a 16 core 5950X.

Hell, it'll be more expensive than just buying a 64-core EPYC server.

and you want to run this Frankenstein monstrosity on a $50 used power supply? (read: Worn the F**K out. No one swaps out a server power supply unless they absolutely HAVE to, meaning used ones are about to die any hour now.

WHY in gods name are you trying to save two or three hundred dollars on what is already a $7000 piece of madness? 4 decent quality 650W 80+ Bronze PSUs from Antec, Silverstone, Seasonic, Be Quiet! or Corsair will be not much more than the pico+Server PSUs you are looking for in the first place.

You now have four Tom's Hardware Moderators telling you its a bad idea. Maybe it's actually a bad idea?
Can you maybe show some basic calculations? Because my calculations show that 4 Ryzen 9 5950 would be significantly cheaper and faster
 

neojack

Reputable
Apr 4, 2019
605
173
5,140
that explains why you want them submerged, it's for the noise. You do crypto at home. it's a good project especially before winter. get serious heat and get paid for it.

why not going for a regular watercooling loop + exterior MO-RA radiators ?
As for the fan, a couple 120v colum fan blowing towards the rads and system is fine. not noisy neither.
 
Last edited:
Oct 23, 2020
25
0
30
that explains why you want them submerged, it's for the noise. You do crypto at home. it's a good project especially before winter. get serious heat and get paid for it.

why not going for a regular watercooling loop + exterior MO-RA radiators ?
As for the fan, a couple 120v colum fan blowing towards the rads and system is fine. not noisy neither.

i did that kind of things with 8 Gpus few years ago. was a good heating for the appartement in canadian winter.

i stopped because the maintenance was too important. not the WC loop, but the linux system and the cards needed reboots etc. but it's was basically free heating and i sold the cards 2 Years later at same price as i bought them. the waterblocks were harder to sell though.


for the pumps, they are good PWM DDCs on aliexpress for like 30$CAD. two of them in series for redondance. powered from a sata port, and monitored on PWM + alert set on motherboard in case of pumpfailure.
no need of a custom top, packaging foam + duct tape, and a piece of tubing between the Inlet and outlet of the pumps. pumps decoupled this way are quiet.

run the waterblocks in parallel to not have an insane pressure drop. chose waterblocks with low restriction.

barb fittings + screw clamps are secure and cheap

best tubing is ZMT

big cylindric reservoir + presure valve on the top (for exemple ,Koolance VLV-VL002K.) that is important on big loops because water level will vary a lot between on and off. this valve let air in when the pumps pulls the pressure from the reservoir.

no fancy coolant, distilled water + ethylene glycol. (ethylene glycol is the main component of some car coolant. check the poison information. if it's says ethylene glycol. i have a galon of 50/50 mix bought in store for 20$ , i mix it at 1 for 3 ratio with distilled water. Its anti corosion too so waterblocks stays shinny ! i wonder why everyone just dont use that. safe for plexi.
No need to try to find a rational point here. It's an engineering project for a little engineer sitting in me. It looks awesome. Much better than normal water loop. It is always as cold as the boiling point of the liquid (as long as you can dissipate the heat to condense it back to liquid). I actually plan to dissipate the heat into the floor heating circuit through the plated heat exchanger but that's a topic for another thread on the build itself. I think I'll create one once I start assembling it. The liquid is on the way already. Boiling pint 58C. Teaser. Tsssss
 

Rogue Leader

It's a trap!
Moderator
Not really. CPU VRM has three 12V lines that are all the same 12V: MB internal, EPS 4pin + EPS 4pin. When the power draw grows the electricity will come from:
a) higher voltage source
b) related to (a): a lower resistance
Power will be sourced from 24pin atx only in one case: 12V line is significantly higher than 12V line of EPS connector (server PSU)
So electrical design is not a gamble. VRM just have 3 connections. And EPS lines are needed because they are thicker and close to VRMs. Nothing more :) Correct me if I'm wrong.

Except I wasn't talking about JUST the CPU but anything related to the operation of a motherboard with a high powered CPU in it. Your assumption is correct EPS connectors are near the CPU specifically due to shorter traces and all that. But the connections are not so black and white, power for the CPU will ALWAYS be sourced from the 24 pin ATX connection first, If whats backing that up is a dinky 120w PSU you will have a bad day.

This is basically true and I agree with that. It's always a safer way. I wanted to see if my assumptions about the wiring all that with one PSU are correct or you can find any flaw besides the nonsense of trying to save another 300$ on such a build

Glad we have an understanding.
 
Oct 23, 2020
25
0
30
But the connections are not so black and white, power for the CPU will ALWAYS be sourced from the 24 pin ATX connection first, If whats backing that up is a dinky 120w PSU you will have a bad day.
What makes you think that the power goes exactly through the ATX 24pin and not EPS? Did you check that with the current clamp? That's relatively easy to do BTW
I think I can do that and post a short vid here in this thread
 

Rogue Leader

It's a trap!
Moderator
What makes you think that the power goes exactly through the ATX 24pin and not EPS? Did you check that with the current clamp? That's relatively easy to do BTW
I think I can do that and post a short vid here in this thread

Did you read what I wrote? Its handled by BOTH. But the CPU socket is primary.

My point was assuming one or the other is incorrect. Again there is no voltage regulation between the two since you're using two different PSUs, and the one connected to your motherboard, is the absolute bare minimum.
 
Oct 23, 2020
25
0
30
Did you read what I wrote? Its handled by BOTH. But the CPU socket is primary.

My point was assuming one or the other is incorrect. Again there is no voltage regulation between the two since you're using two different PSUs, and the one connected to your motherboard, is the absolute bare minimum.
WDYM primary? they are physically connected. They are the same 12V rail in your computer. Just wired through different paths. And the current will flow through whatever path has lower resistance. Meaning that there is no primary and secondary path.
Now if we have Pico powering ATX24pin and Server PSU powering 4+4 CPU EPS connector what you say can be either true or false depending on where the 12V will be higher. And technically it can be higher on the 24pin side and in that case Pico might end up overloaded. Though now I see there are pico PSUs out there that just pass through the 12V line without any DC-DC on it. I suspect they need to invert that 12V line for -12V pin though
 

Rogue Leader

It's a trap!
Moderator
WDYM primary? they are physically connected. They are the same 12V rail in your computer. Just wired through different paths. And the current will flow through whatever path has lower resistance. Meaning that there is no primary and secondary path.
Now if we have Pico powering ATX24pin and Server PSU powering 4+4 CPU EPS connector what you say can be either true or false depending on where the 12V will be higher. And technically it can be higher on the 24pin side and in that case Pico might end up overloaded. Though now I see there are pico PSUs out there that just pass through the 12V line without any DC-DC on it. I suspect they need to invert that 12V line for -12V pin though

Thats exactly what I mean, the power is coming from different places going to basically the same place, but the VRMs don't know that. There is no "12v rail" on the board, Rails are in PSUs, there are circuits and traces on the motherboard which are separated by where they are connected. There are primary and secondary paths, EPS is considered a secondary path, the board manages this power in conjunction with the PSU. By using 2 separate sources you've eliminated this management. Suffice to say under low loads little is drawn via the EPS. Under the heaviest loads once the socket is used primarily the EPS supplies the most power

You are correct that current will flow on the lowest resistance path, but this will cause the system to attempt to overdraw the Pico PSU. You will burn it up, and maybe damage the CPU or board in the process.
 
Solution
Oct 23, 2020
25
0
30
Thats exactly what I mean, the power is coming from different places going to basically the same place, but the VRMs don't know that. There is no "12v rail" on the board, Rails are in PSUs, there are circuits and traces on the motherboard which are separated by where they are connected. There are primary and secondary paths, EPS is considered a secondary path, the board manages this power in conjunction with the PSU. By using 2 separate sources you've eliminated this management. Suffice to say under low loads little is drawn via the EPS. Under the heaviest loads once the socket is used primarily the EPS supplies the most power

You are correct that current will flow on the lowest resistance path, but this will cause the system to attempt to overdraw the Pico PSU. You will burn it up, and maybe damage the CPU or board in the process.

I see what you mean but I think you're wrong about the EPS being secondary just because EPS 12V path is the one with lower resistance. "There are primary and secondary paths" - there may be a thousand paths to one source of 12V. The current will flow the one with less resistance. If one is thin (like the mobo traces) it will always be secondary.
I'm gonna capture that on video with/without load using current clamps.
 

Rogue Leader

It's a trap!
Moderator
I see what you mean but I think you're wrong about the EPS being secondary just because EPS 12V path is the one with lower resistance. "There are primary and secondary paths" - there may be a thousand paths to one source of 12V. The current will flow the one with less resistance. If one is thin (like the mobo traces) it will always be secondary.
I'm gonna capture that on video with/without load using current clamps.

Again VRMs control this, there is circuitry that controls this both on the board and the PSU. You would be correct if there were direct lines from the PSU to the CPU but they are not direct. The voltage is reduced into the CPU. I'm not saying you're going to fry the CPU either, but you very much could damage the board and the PICO PSU trying to do this. You can measure with clamps all you want, its not going to show what you want to see on the 12v side, what is happening is within the board.
 
Oct 23, 2020
25
0
30
Again VRMs control this, there is circuitry that controls this both on the board and the PSU. You would be correct if there were direct lines from the PSU to the CPU but they are not direct. The voltage is reduced into the CPU. I'm not saying you're going to fry the CPU either, but you very much could damage the board and the PICO PSU trying to do this. You can measure with clamps all you want, its not going to show what you want to see on the 12v side, what is happening is within the board.
One segment is from PSU to VRM. This is one common 12V routed through different paths: 24pin ATX and mobo traces and EPS 4+4. We're not interested in anything past VRM because PSU has nothing to do with it. Talking about the 12V part: current will be sourced from whatever source with higher voltage. In a single PSU setup voltage on the EPS 4+4 will always be higher (because of lower resistance and in turn lower voltage drop). So the main source is not ATX 24pin connector but EPS.
In the case where Pico PSU passes through the 12V line (only some of them do that), it's not an issue because in that case 24pin 12v and EPS 12v are the same 12v. If the Pico has DC-DC circuit to stabilize 12v it can boost voltage and in that case, might become the source of power for the entire CPU leaving EPS lines unloaded. That's a very possible scenario and in that case, Pico PSU would be fried for sure.
 
Oct 23, 2020
25
0
30
We agreed earlier that it is not worth it to go this way. Further discussion was just on powering the CPU VRM, not really about whether or not to go this way in my particular case.
But thanks