Server MB with 4 x16 PCI-E Slots?

[h.caspersen]

Hi All

I am building a all in one ESXi server (Plex, Xpenology or Unraid and a personal Remote desktop pc)
Sofar the part purchased are: PCPARTPICKER

I want the new chipset and the new features that comes with it, but I also need allot of PCI-express with 2 x 8X, 1 x 16X, and 2 x 4X (Depending if the MB has 10GB ethernet?)

The two candidates I have found sofar? are these two boards:

ASUS WS Z390 PRO - This has a spare 4X for a ASUS 10GB ethernet card
Supermicro C9Z390-PGW (With 10GB ethernet)

Any input to what is best? pros cons is most welcome!

Also any recommendations to a CPU? (Intel) (The P2000 will handle the Plex transcoding) so I don't need a 9900K for this, and I want a "okay" idle power consumption because it's going to run 24/7 and power is expensive here

Thanks looking forward to your input

Regards
Casperse

 

[kanewolf]

Although the motherboards you listed have four x16 physical slots, they operate as x8 if they are all in use.

 

[h.caspersen]

Although the motherboards you listed have four x16 physical slots, they operate as x8 if they are all in use.

Isn't this only the case if you are using them with graphic cards in a SLI mode?
I am not using them like that so I don't think that restraint is there?

there is also a fifth slot for a PCI-E 4x so even if this was a limitation, then I would have 16x,8x,8x,4x which would fit the cards I have listed

 

[karenjoly]

The Wolf is correct.

I thought you said you need PCI-express with 2 x 8X, 1 x 16X, and 2 x 4X ?

The ASUS won't do that config. The board will be one x4 short. To wit.,

4 x PCIe 3.0 x16 (single at x16, dual at x16/x16 mode, triple at x16/x8/x8 mode or quad x8/x8/x8/x8 mode)
1 x PCIe x4 (max at x4 mode)

My suggestion is to suffer the gpu running x8 which will allow it to perform well. X16 for gpu's are not an absolute requirement.

Use all four x16 slots : The gpu ; the two x8 devices; and one of the x4 devices in the last x16 which will be set to x8 and so will run the x4 card.

Then the 2nd x4 card will go into the x4 slot.

 

[h.caspersen]

Okay I just thought it was a SLI limitation (So many reviews states 4 x 16X PCI-E)

Correct the last 4X was for the missing 10 Gb ethernet on the Asus board
The other board will provide me with native 10 Gb support, so that would require one less 4x socket

but do you have any input to what board is best? or another board recommendation?
Asus seems to have the best Bios/UI but Supermicro (still on bios 1.0) and is known for their server MB?

 

[kanewolf]

If you want PCIe lanes, you should look at a Threadripper. Many more PCIe lanes than Intel.

 

[karenjoly]

I can't say, never used servers, but I use ASUS boards with no complaints.

 

[h.caspersen]

If you want PCIe lanes, you should look at a Threadripper. Many more PCIe lanes than Intel.

Threadripper and ESXi is a really bad mix!, also intel is the only CPU having M2 working as a boot on ESXi

 

[h.caspersen]

I can't say, never used servers, but I use ASUS boards with no complaints.

Yes ASUS is more expensive and dont have 10GB ethernet but I have the same feeling that it's the safest choice

 

[h.caspersen]

Ok looking for more PCI lanes and I found this for the 7900X CPU's but it looks like the motherboard chipset dosent support as many PCI-express streams as the above one using the "commercial" Z390 chipset is that correct?

https://www.asus.com/dk/Motherboards/PRIME-X299-DELUXE/specifications/ using the X299 Chipset?
Thats really weird considering that this CPU supports 44 PCI-E lanes instead of the 16 above?

 

[karenjoly]

Most modern intel chipsets support 24 lanes.

Intel processors' support, however, will vary between 16 and 44 lanes.

Using the x299 with 7980x is 24 + 44 = 68 lanes.

The z390 supports 24 lanes and the processors on that chipset have 16 lanes =40.

Don't confuse the "expansion slot" lanes with the total lanes available on a motherboard.

People only care about the # of cpu lanes available trying to avoid connecting to chipset lanes to reduce lag.

Like they would notice !

 

[h.caspersen]

Most modern intel chipsets support 24 lanes.
Intel processors' support, however, will vary between 16 and 44 lanes.
Using the x299 with 7980x is 24 + 44 = 68 lanes.
The z390 supports 24 lanes and the processors on that chipset have 16 lanes =40.
Don't confuse the "expansion slot" lanes with the total lanes available on a motherboard.
People only care about the # of cpu lanes available trying to avoid connecting to chipset lanes to reduce lag.
Like they would notice !

Yes I am starting to see that....
I went on looking at the new Xeon E-2100 series (because that would also be on the ESXi support list) and now with quicksync and ECC I found that they use a motherboard with a chipset called C246 and with 6 expansion slots! but to my surprise it's even more limited than the Z390 chipset supporting 24 lanes?

https://www.asus.com/us/Motherboards/WS-C246-PRO/specifications/
Total Slot : 6
Slot Location 1 : PCI-E x1 (Gen2 X1 Link)
Slot Location 2 : PCI-E x16 (Gen3 X16 + X8 Link)
Slot Location 3 : PCI-E x16 (Gen3 X8 Link)
Slot Location 4 : PCI-E x1 (Gen2 X1 Link)
Slot Location 5 : PCI-E x16 (Gen3 X4 Link)
Slot Location 6 : PCI-E x4 (Gen1 X4 Link)

I just can't find a MB and a CPU that would cover my needs if I have to get ECC memory support also? (Someone told me that would be best for storage usage), and the expansion cards listed above it seems impossible.

 

[kanewolf]

For what you want, you have to look at more expensive CPUs. The Xeon W series CPUs have 48 PCIe lanes for example. OR you have to look at dual socket implementations. Dual socket motherboards have PCIe lanes from two CPUs to spread around.

 

[h.caspersen]

Most modern intel chipsets support 24 lanes.
Intel processors' support, however, will vary between 16 and 44 lanes.
Using the x299 with 7980x is 24 + 44 = 68 lanes.
The z390 supports 24 lanes and the processors on that chipset have 16 lanes =40.
Don't confuse the "expansion slot" lanes with the total lanes available on a motherboard.
People only care about the # of cpu lanes available trying to avoid connecting to chipset lanes to reduce lag.
Like they would notice !

If I understand this correctly then my only option would be to go with Xeon W serie (Like suggested by kanewolf)
None of the boards/CPU's below would be able to run with the below cards (Right?)
8X (16X) Quadro P2000 card
8X - LSI SAS9201-16i PCI-Express 2.0 x8 SAS 6Gb/s HBA Card
8X - LSI SAS9201-8i PCI-e Controller / 9211-8i (IT-mode)
4X Intel Pro 1000 VT Quad Port NIC - PCI Express
= 28 Lanes?

I was so happy to see that the Xeon E-2100 serie would provide me with a Xeon and the GPU 630 but looking at the CPU and motherboard it looks like this cant supply me with enough lanes? (16 Pci lanes)

https://www.gigabyte.com/Server-Professional-Products/C246-WU4-rev-10#kf

1 x PCI Express x16 slot, running at x16 (PCIEX16)
* For optimum performance, if only one PCI Express graphics card is to be installed, be sure to install it in the PCIEX16 slot.
1 x PCI Express x16 slot, running at x8 (PCIEX8)
* The PCIEX8 slot shares bandwidth with the PCIEX16 slot. When the PCIEX8 slot is populated, the PCIEX16 slot operates at up to x8 mode.
2 x PCI Express x16 slots, running at x4 (PCIEX4_1/PCIEX4_2)
* The PCIEX4_2 slot shares bandwidth with the M2M connector. The PCIEX4_2 slot becomes unavailable when a PCIe SSD is installed in the M2M connector.
(All of the PCI Express slots conform to PCI Express 3.0 standard.)
1 x PCI slot

https://www.asus.com/Motherboards/WS-C246-PRO/
Total Slot : 6
Slot Location 1 : PCI-E x1 (Gen2 X1 Link)
Slot Location 2 : PCI-E x16 (Gen3 X16 + X8 Link)
Slot Location 3 : PCI-E x16 (Gen3 X8 Link)
Slot Location 4 : PCI-E x1 (Gen2 X1 Link)
Slot Location 5 : PCI-E x16 (Gen3 X4 Link)
Slot Location 6 : PCI-E x4 (Gen1 X4 Link)

So only a Xeon W would accommodate these cards right?

Sorry for being a little slow