[SOLVED] PCIe Lanes Again: GPU, Video Capture, RAID

[Gordon Fecyk]

I have a system with a Core i9 10th gen CPU and MSI MPG Z490 motherboard. I also have a NVME storage device, a PCIe Aver Media video capture card, and a PCIe Firewire card - I still have some old video cameras using that. All of these used to work with an RTX 2070 graphics card. When I replaced the 2070 with a 3070 Founder's Edition card, the PC would stutter on startup and never really boot. I had to remove the capture card to make the thing usable again, and I've removed the Firewire card too.

I've since learned that modern GPUs are PCIe lane hogs, and that this is the cause of my stuttering interaction. I have the GPU in the first (closest to the CPU) slot so presumably it's using CPU-direct PCIe lanes. BIOS Setup tells me I have two options for PCIe lane management: 1 x 16x, or Auto. At least this configuration is working, and it's working better since applying a 2022 BIOS update. There's a whole story about video editing mayhem that ended up mostly being solved with the BIOS update.

So, I can't add all of these cards back, at least not on this system. On other, older systems with older gen GPUs this capture card still works. Using an older system to capture video is an option, but now I'm thinking it's time to build a new PC. While I'm at it, I'm even considering a RAID controller instead of using the onboard NVME or SATA ports and one of those would take up eight lanes of its own - I'm dealing with 4k video, and lots of it, as of late.

I've done a lot of reading, even considering building a dual processor system just to get the PCIe lane count up. And that's where Intel ARK is messing with me - I'm reading that the Z790 (Core i9 13th gen) and C621 (Xeon Scalable) chipsets only support 20 lanes, but I'm reading elsewhere that's chipset lanes and not CPU lanes - the CPU will have PCIe lanes that bypass the chipset. What I don't know is if the chipset lane limitation imposes a bottleneck on the CPU lanes - if a CPU supports more PCIe lanes than the chipset, can these be used?

I'd love to cram an RTX 3000 GPU, the PCIe x8 RAID card, the PCIe capture card, and the PCIe Firewire card back into one system. I wouldn't have any NVME devices added. That's still 26 PCIe lanes used.

If I were building a 'normal' system I'd stick with just two devices, a modern GPU and a NVME device, using 20 lanes total. I've recently wondered what happens when piling SATA and USB3 devices on top of this though, if they consume PCIe lanes as well.

 

[USAFRet]

1. Why RAID, what type array, and with what drives? Rarely is RAID a benefit.
2. Dual CPU only counts if your software can utilize it. Very little outside of server level things do.

 

[Gordon Fecyk]

The drives would be spinning rust just to get more storage on the cheap. I already have a hybrid drive on SATA 6 Gb for this purpose, plus File History backups, but after graduating from 1080p to 4k I'm finding this isn't quick enough. I also use VEGAS 20 Pro as my editor. I'm otherwise aware that RAID doesn't offer much benefit anymore, aside from some protection from hardware failure when using RAID 5 or 6.

VEGAS 20 does support multiple threads for editing and rendering now, so it should take advantage of multiple CPUs. Otherwise, the only reason I even considered dual CPUs was to get more PCIe lanes.

 

[USAFRet]

The drives would be spinning rust just to get more storage on the cheap. I already have a hybrid drive on SATA 6 Gb for this purpose, plus File History backups, but after graduating from 1080p to 4k I'm finding this isn't quick enough. I also use VEGAS 20 Pro as my editor. I'm otherwise aware that RAID doesn't offer much benefit anymore, aside from some protection from hardware failure when using RAID 5 or 6.

VEGAS 20 does support multiple threads for editing and rendering now, so it should take advantage of multiple CPUs. Otherwise, the only reason I even considered dual CPUs was to get more PCIe lanes.

Multiple threads is not the same as multiple CPUs.

And even a SATA III SSD is faster than HDD + RAID 0.
Use that for your working drive, offload to HDD after.

 

[Gordon Fecyk]

Multiple threads is not the same as multiple CPUs.

If I can get more PCIe lanes out of a single CPU system, that's good too.

 

[USAFRet]

If I can get more PCIe lanes out of a single CPU system, that's good too.

Thats more a function of the motherboard.

 

[Gordon Fecyk]

And

Thats more a function of the motherboard.

And this is where I'm getting stuck, trying to find a board with more lanes. It's looking like modern boards have just enough expansion room for one NVME device and one GPU. Some Micro-ATX boards even ship with just one PCIe slot now.

As a consequence, I can't capture video from my PCIe capture card anymore. Sure, there are USB capture devices, but none that do HDMI 1080p. And I'd rather not resort to a USB capture solution if I can help it. The RAID card idea was just 'out there,' wondering if it was doable.

 

[USAFRet]

And


And this is where I'm getting stuck, trying to find a board with more lanes. It's looking like modern boards have just enough expansion room for one NVME device and one GPU. Some Micro-ATX boards even ship with just one PCIe slot now.

As a consequence, I can't capture video from my PCIe capture card anymore. Sure, there are USB capture devices, but none that do HDMI 1080p. And I'd rather not resort to a USB capture solution if I can help it. The RAID card idea was just 'out there,' wondering if it was doable.

Just at random, the Rog Maximus XIII
https://dlcdnets.asus.com/pub/ASUS/..._XIII_EXTREME_GLACIAL_One_Page_Spec_Final.pdf

5x M.2 ports
2x PCIe 4.0 CPU and 1x PCIe 3.0 chipset.

 

[Gordon Fecyk]

The Z590 chipset of the ROG XIII has a similar problem as everything else:

Supported Processor PCI Express Port Configurations: 1x16+1x4 or 2x8+1x4 or 1x8+3x4. And the GPU will try to hog 16 lanes making the 1x16 + 1x4 the only running configuration.

 

[USAFRet]

What hardware do other people that use this software use?

What hardware does the software publisher recommend?

 

[Gordon Fecyk]

Why can't I just get an answer to the question of getting more PCIe lanes out of a PC?

I have a unique use case, but I thought I described it with enough detail to get a useful answer. This is a video editing and video capture PC I'm trying to either fix, or build anew. The problem is a modern graphics card and a modern video capture card can't coexist on a consumer motherboard, at least not on mine, and apparently not on anything current, either.

Anyone else here able to take a shot at this problem?

 

[4745454b]

To answer your question, for Intel you need the Xeon W CPUs.

https://www.intel.com/content/www/us/en/products/details/processors/xeon/w.html

If you go with AMD it's a bit odd for the consumer side.

https://imgur.com/a/b7tntpZ

View: https://imgur.com/a/b7tntpZ


There are the normal 16x lanes for the PCIe 16x slot which go right to the CPU. In theory there are more lanes attached to the chipset, but they then hit a bottleneck going to the CPU in that 4x bit. So there is 8x lanes attached to the chipset for you to use, but they will then be forced into a 4x but to get to the CPU. It's my understanding if you move to their EPOC chips there are many more lanes, but that's beyond what I know.

 

[Rogue Leader]

The Z590 chipset of the ROG XIII has a similar problem as everything else:

Supported Processor PCI Express Port Configurations: 1x16+1x4 or 2x8+1x4 or 1x8+3x4. And the GPU will try to hog 16 lanes making the 1x16 + 1x4 the only running configuration.

What you're looking for is not available in normal consumer chipsets.

Take the current 13th gen Intel processors, they have 16 PCIe 5.0 (or 4.0) lanes that will go to the GPU, and 4 additional 4.0 lanes dedicated to the NVME slot. Even if you didn't use that slot, 4 lanes does nothing for your problem. The Z790 Chipset is rated at 28 lanes, but all 28 are not necessarily available to you even if you don't have everything plugged in. Some are dedicated to SATA ports, some to other motherboard parts. Even the top Z790 board only provides you with 2 slots that will run at PCIe 4.0 x16, one goes direct to the CPU, 1 goes through the chipset. If you try to run 2 cards in the 2 CPU connected slots they run at x8.

As suggested above if you want to do this you need to look into moving to a HEDT platform such as an Intel XEON, or an AMD Ryzen Threadripper PRO. Those types of processors offer many different configurations including up to 128 PCIe 4.0 lanes that go straight to the processor.

Unfortunately this jacks the cost up considerably. If I were you I would reevaluate my workflow to see if I could eliminate or combine some devices to limit myself to using a consumer level Intel 13th gen processor and Z790.

 

[Gordon Fecyk]

As suggested above if you want to do this you need to look into moving to a HEDT platform such as an Intel XEON, or an AMD Ryzen Threadripper PRO. Those types of processors offer many different configurations including up to 128 PCIe 4.0 lanes that go straight to the processor.

OK, at least that's a potential solution. I already have questions out to some board maker forums and am looking at Asustek's WS 621E SAGE board plus some used CPUs. Looks like I can use 1st gen Xeon scalable to start and graduate to 3rd gen when budget allows. I'll try to be careful when sourcing CPUs.

 

[Rogue Leader]

OK, at least that's a potential solution. I already have questions out to some board maker forums and am looking at Asustek's WS 621E SAGE board plus some used CPUs. Looks like I can use 1st gen Xeon scalable to start and graduate to 3rd gen when budget allows. I'll try to be careful when sourcing CPUs.

That will work, however its super excessive for what you want. It sure sounds like 64 PCIe lanes would be far more than enough for you, which can be done with 1 mid range Xeon or Threadripper.

That said you can probably get these used for very reasonable prices. Just be aware that you may be disappointed in its performance compared to what you have now. Your current CPU is many times faster in single and low threaded tasks. So like using your video software, Windows, tasks etc will all be noticeably slower.

I'm not a fan of buying something to start only to upgrade in a short time, this board doesn't support 3rd gen + so you'd need a new board and CPUs, etc, why not just do it right the first time, and have a system that could work fine for you for 10 years or more. Not saying to buy the newest most expensive stuff, but look further into newer or 1 gen old hardware and what you can get and build that will be your "final" system. It will likely make more financial sense.

 

[Gordon Fecyk]

however its super excessive for what you want. It sure sounds like 64 PCIe lanes would be far more than enough for you, which can be done with 1 mid range Xeon or Threadripper

I was looking at Xeon W as a potential solution, but kept getting tripped up on the C621 chipset having only 20 lanes. After more reading though, that sounds like it'd be chipset lanes only and there'd be at least 16 lanes from the CPU to a x16 card slot then, leaving the chipset lanes open for other things. This could work, if so.

 

[Rogue Leader]

I was looking at Xeon W as a potential solution, but kept getting tripped up on the C621 chipset having only 20 lanes. After more reading though, that sounds like it'd be chipset lanes only and there'd be at least 16 lanes from the CPU to a x16 card slot then, leaving the chipset lanes open for other things. This could work, if so.

The chipset lanes do not matter in this scenario, ignore them.

Using the board you posted earlier, even if you bought a cheap Xeon Bronze 3104 CPU (which is $40 on ebay right now), it will provide to that board and slots 48 PCIe lanes. If you ran 2 of them in that board, youd have 96 lanes. Plus tghe additional 20 (of which probably 16 are available) from the chipset. Now I would not reccomend that CPU at all, the cores are low clocked, its made for a server really, but you can understand where I am going here.

Now moving to Xeon W-3xxx processor those have 64 PCIe lanes to provide to whatever board you get. W-1xxx and W-2xxx processors have 48. Xeon W processors are a far better than the older tech Bronze, Gold, Platinum, etc processors. They have significantly faster cores which should provide you the performance you expect from a modern system in daily tasks. Keep in mind even though the board you posted earlier has the C621 chipset it DOES NOT support Xeon W processors. Its important to keep those compatibility lists in mind before buying a board.

 

[Gordon Fecyk]

Keep in mind even though the board you posted earlier has the C621 chipset it DOES NOT support Xeon W processors.

At least we have an option. Not with the C621 I guess, but there are C621A boards out there. There are quite a few Xeon W pulls from Mac Pros and some relatively inexpensive Xeon Silvers available.

Also at least, I can expect that these systems will offer independent CPU lanes vs chipset lanes and the chipset doesn't bottleneck the CPU lanes. Looks like we have a solution!

 

[Rogue Leader]

At least we have an option. Not with the C621 I guess, but there are C621A boards out there. There are quite a few Xeon W pulls from Mac Pros and some relatively inexpensive Xeon Silvers available.

Also at least, I can expect that these systems will offer independent CPU lanes vs chipset lanes and the chipset doesn't bottleneck the CPU lanes. Looks like we have a solution!

Yep like I said though just be careful of clock speed on the cheap Xeon Silver processors, anything under 3 ghz is going to feel slow, its running older Skylale SP cores. IMO you're far better off with the XEON W.