Question do I have enough PCIe lanes?

[sspeed]

I am using an Asrock B660M-ITX/ac motherboard.

I have 4 SATA SSDs, 1 NVMe SSD. Nic is Intel(R) 82599 10 Gigabit Dual Port. Also has an onboard Intel I219-V nic. It's a homelab esxi.

From my understanding. the 82559 nic takes 8 chipset PCIe lanes. What gen though? Each SATA takes 1. But what gen? NVMe has 4 to the CPU. I assume the 82559 takes gen3 lanes and the SATA as well?

Either way, I seem to be limited by the B660 with 6 PCIe 4.0 ports and 8 PCIe 3.0 ports? Between the 4 SATA SSDs and 82559 nic I am using 12 Gen3 PCIe lanes? Just wondering if I should look for a used H670 or Z690 board? Or do the 6 PCIe 4.0 lanes function as 3.0 when nothing else is on them?

 

[SkyNetRising]

From my understanding. the 82559 nic takes 8 chipset PCIe lanes. What gen though?

82559 nic works in PCIE 2.0 mode.

Each SATA takes 1. But what gen?

No. Sata goes through chipset.

NVMe has 4 to the CPU.

Yes.

I assume the 82559 takes gen3 lanes and the SATA as well?

No. PCIE slot is connected to cpu PCIE lanes. But 82559 works in PCIE 2.0 mode.

Either way, I seem to be limited by the B660 with 6 PCIe 4.0 ports and 8 PCIe 3.0 ports?

What? Please elaborate.

 

[sspeed]

Apologies, excuse my ignorance. I was going by this and trying to make heads or tails.

https://www.tomshardware.com/news/intel-h670-b660-h610-motherboard-roundup

It shows 6 PCIe 4.0 lanes and 8 PCIe 3.0 lanes. I don't see any PCIe 2.0? Those lanes are just for the chipset? And 82559 goes to the CPU, so chipset lanes shouldn't matter? i7-12700 if that matters.

 

[SkyNetRising]

It shows 6 PCIe 4.0 lanes and 8 PCIe 3.0 lanes. I don't see any PCIe 2.0? Those lanes are just for the chipset? And 82559 goes to the CPU, so chipset lanes shouldn't matter?

I don't understand your concern.
What is not working for you?

Intel(R) 82599 is PCIE 2.0 device in PCIE 4.0 slot. It works in backwards compatible PCIE 2.0 mode.

 

[sspeed]

I added a Gen4 NVMe, expecting it to be snappier. ESXi backups to a Synology 1817+ are actually taking longer than when the VMs were on a SATA SSD. Can't figure out why, so wondering if ran out of lanes.

 

[SkyNetRising]

I added a Gen4 NVMe, expecting it to be snappier.

What model M.2 drive?

 

[sspeed]

WD Black SN850X. CrystalDiskMark is showing expected performance, but backups over the 10gbit wire are definitely slower, about 20 min slower on a 2.5hr backup.

 

[SkyNetRising]

At what data rate are backups happening?

You'll be limited either to sata speed or network speed.

 

[sspeed]

Backing up at about 1.5gbps to a Synology 1817+ (10g nic as well). 197GB in about 20min, roughly. Slightly slower from the NVMe vs the SATA SSD.

 

[Eximo]

Could be a few things, but I suspect it is bandwidth limitations through the chipset vs through the CPU DMI. Probably just a little more overhead to handle the traffic while still running everything else in the system.

In answer to your other question, lanes are fixed, and can't be reallocated unless indicated in the motherboard manual. PCIe speed is variable.

So even though you have 20 lanes from the CPU, in this scenario you are only using 12 lanes. All other lanes are being provided by the chipset (14), and that has a 4x lane connection to the CPU.

So to get data from the NVMe drive from the NIC, no problem, via the CPU. To go from your NVMe drive to anything else in the system, also has to compete with all the other devices in the system.