Question PCIe M.2 Adapter Board in an X99 System ?

[dcairns]

So I want to add some more fast M.2 drives on my X99 system and the PCIE Adapter boards seem the obvious route. My board supports one M.2 drive, but I would like to be able to add more. So I have poked around, reading/watching stuff on how this works, and looks like it would be fine on my system, so I am looking for a check on my research here.

So my idea is a something like the ASUS Hyper M.2 X16 PCIe 4.0. (open to other/cheaper suggestions) Yes, I know my system only does PCIE 3.0, so I might be over spending on that, but being able to add more M.2 NVME drives as I can afford them, is an interesting idea. The idea would be to have the drives in the ASUS Hyper board appear as one drive or separate drives, no RAID.

My understanding of PCIE lane usage:

i7-6900K supports 40 PCIE lanes

Asus X-99 A II Motherboard supports PCIE 3.0 and 40 PCIE lanes.

The manual rather vaguely breaks down the PCIE slot/lane support like this:

GPU Cards PCIe Slot 1 PCIe Slot 3 PCIe Slot 4

1 x16

2 N/A x16 x16

3 x8 x16 x8

So by my understanding, the M.2 slot takes x4, the 1070 Ti takes 16 making at total of 20 lanes in use, with 20 more free. A ASUS Hyper M.2 X16 PCIE 4.0 with 4 SSD drives in it would take an additional 16 leaving me with 4 remaining.

Just to show how much my work drive could use some help,

Crystal Mark rates my M.2 boot drive as: Seq Q32T1 1810/1254

My SATA SSD work drive as: Seq Q32T1 528/507

My SATA mechanical HDD as: Seq Q32T1 217/211


System specs::

Windows 10 22H2 19045.4651 (never going to 11, planning on jumping ship to Linux)

i7-6900k @4Ghz

Asus X-99 A II Motherboard

Water cooled CPU, 96GB RAM

NVIDIA GeForce GTX 1070 Ti (8Gb) feeding three 27" monitors

Storage:

One built in M.2 NVME slot in use as the boot drive

2TB SSD Drive (SATA)

1TB SSD Drive(SATA)

Three spinning 8 TB Drives (SATA)

12TB spinning 12 TB Drive (SATA)

Offline 8TB backup drive (USB 3.1) for essentials.

The system is used for photo and video editing workstation general PC use and some gaming. Yes, I am a digital pack rat.

 

[dcairns]

Well, I went ahead and got the ASUS Hyper M.2 X16 PCIe 3.0 X4 Expansion Card V2 and a SAMSUNG 990 PRO SSD 4TB PCIe 4.0 M.2. So I will just find out on my own. I am pretty sure it will work just fine. Perhaps I overspent on the M.2 drive, but 4.0 will give it some future life.

 

[Maxxify]

You need single-slot PCIe bifurcation to make use of the Hyper. It's not the solution you want for that machine. The manual actually isn't vague, but to elaborate, the 6900K is a 40-lane CPU so for the 3.0 PCIe slots can do x16, x16/x16, or x8/x16/x8. You can use one of these for a discrete GPU and another for an M.2 expansion card that has a PCIe switch/controller on it. Choice of AIC depends on what kind of bandwidth you want. If you're going for the maximum x16 3.0 with four drives, then it's something like the SSD7104/SSD7105. If x8 3.0 is enough or you want to use all three PCIe slots with the x16 for the GPU, then the SSD7204. (just examples, there's probably better options)

 

[dcairns]

Well, this is the kind of info I needed before getting impatient and pressing the "Buy" button . I was able to cancel the order for the Asus add in card. After reading your post and some more "learning", I realized the Asus card was all about RAID 0 (striping) and there is no mention of bifurcation in my motherboard documentation. I can appreciate the performance of doing that, but having tried RAID ) 20 years ago, I am not fond of it. Too temperamental, but maybe it is better now. Anyways, I am just looking to get the best performance I can out of the M.2 SSD. So back to looking at add in card to carry the M.2. Maybe just the single M.2 would be ok.

 

[Maxxify]

Well, this is the kind of info I needed before getting impatient and pressing the "Buy" button . I was able to cancel the order for the Asus add in card. After reading your post and some more "learning", I realized the Asus card was all about RAID 0 (striping) and there is no mention of bifurcation in my motherboard documentation. I can appreciate the performance of doing that, but having tried RAID ) 20 years ago, I am not fond of it. Too temperamental, but maybe it is better now. Anyways, I am just looking to get the best performance I can out of the M.2 SSD. So back to looking at add in card to carry the M.2. Maybe just the single M.2 would be ok.

Technically, your board does bifurcation across multiple PCIe slots, but for that type of card you need it do it with/within a single slot. Many consumer boards can do this but with HEDT like yours you have a lot more CPU lanes to work with so you can use something higher-end. You just need an AIC with hardware on the PCB, most commonly a PCIe or packet switch. These AICs cost more.

RAID for either type of AIC is not necessary at all. In most cases, it would be software RAID, anyhow, not hardware RAID. So it's not correct to say the Hyper or my suggested alternative would force you into RAID of any kind. If your goal is to add up to four drives with as much bandwidth as possible, and bandwidth is only one aspect of performance mind you, then you would need to take my suggestion.

 

[thestryker]

You basically have two options here:

1. Straight PCIe to M.2 adapter which is a 1:1 so you can add a single M.2 drive into a single PCIe slot
2. An adapter with a PCIe switch on it which will handle the bifurcation on the card itself. The cost would depend on how many drives you wanted supported and what amount of bandwidth you wanted allocated

example of the first: https://sabrent.com/products/ec-pcie
example of the second: https://www.newegg.com/riitop-dul-nvtpce8x-pci-express-controller-card/p/17Z-0061-00097

 

[dcairns]

Well, those HighPoint 7104 or 7105 are a bit pricey for now. So I just punted and got a single SABRENT NVMe M.2 SSD to PCIe X16/X8/X4 Card for $18. I can defer the idea of a multi SSD card for later when I can afford more M.2 SSDs. This will out perform the existing 2TB SATA SSD that I use at the moment. And maybe those nicer multi M.2 cards will get cheaper by then.
Thank you for the help

 

[Bob.B]

Well, those HighPoint 7104 or 7105 are a bit pricey for now. So I just punted and got a single SABRENT NVMe M.2 SSD to PCIe X16/X8/X4 Card for $18. I can defer the idea of a multi SSD card for later when I can afford more M.2 SSDs. This will out perform the existing 2TB SATA SSD that I use at the moment. And maybe those nicer multi M.2 cards will get cheaper by then.
Thank you for the help

Depending on what you do with the pc don't be surprised if you don't see much a diff in perf after you do this swap.

 

[dcairns]

Since I overspent on the 4TB M.2 SSD, for the moment I am going to try the single M.2 SSD on the single adapter card. The card was cheap enough ($18). However given the cost of adding more three more matching 4TB M.2 SSD's, I may just do a two M.2 SSD card. That would keep the add in card cost down and only need to buy another 4TB M.2 SSD for a 2x speed increase.
I am still struggling with the proper way to vet the add in cards for the bifurcation.
My use case for this "drive" will be photo editing and video editing, so large sequential read/write activity.

 

[USAFRet]

and only need to buy another 4TB M.2 SSD for a 2x speed increase.

You're thinking of a RAID 0 with these?

Apart from artificial benchmarks, RAID 0 does not benefit like it did with HDDs.

In some instances, it is even slower.

 

[dcairns]

Perhaps I misunderstood the recommendation for the SSD7104/SSD7105 add in cards. I thought their performance boost was due to the RAID 0 striping.

 

[thestryker]

I am still struggling with the proper way to vet the add in cards for the bifurcation.

PCIe 3.0 cards will all be using ASM2812 or ASM2824 controllers for the most part. These should be available starting at around $100 USD, but potentially less if you're willing to order direct from China via AliExpress or the like. I linked an example in my prior post.

PCIe 4.0+ ones you're probably talking a significant investment that doesn't make sense. PCIe switches are very expensive so I'd expect $300+ for pretty much anything in this category.

 

[USAFRet]

Perhaps I misunderstood the recommendation for the SSD7104/SSD7105 add in cards. I thought their performance boost was due to the RAID 0 striping.

Traditionally, yes. RAID 0 + HDD was a BIG performance boost, in the right circumstances.

But with the advent of SDDs, not so much.
Especially in typical consumer use.

SSDs main benefit over HDD is the near 0 access time. That is across all SSDs.
Small file access, which is what we do the vast majority of the time, benefits not at all from that RAID 0.

Often, even just between the different PCIe variants.

In a blind test, it is often hard to tell the user facing difference between PCIe 3.0, 4.0, 5.0 drives.

Now...if you were routinely transferring large blocks of data between 2 such RAID 0 arrays, you'd see a difference. Potentially a big diff.
But that is not what we do.

If RAID 0 + SSD were such a big performance boost, we'd all be doing it.

Here is a couple of older tests, showing basically NO difference.
I've not seen anything to change my mind.
http://www.tomshardware.com/reviews/samsung-950-pro-256gb-raid-report,4449.html
http://www.tomshardware.com/reviews/ssd-raid-benchmark,3485.html

 

[dcairns]

Yes, I have been looking at that RIITOP Dual NVMe PCIe Adapter you linked earlier. That is a lot more affordable than the SSD7104/SSD7105 option. And perhaps fits my situation better at this point.

Looking over those RAID links I can see the mixed results of RAID 0 and it got me to do a test on my existing system. This highlights the issue I am trying to resolve. A 664KB Photoshop file takes 29 seconds to save to my existing SATA SSD
Strangely it takes the same time to save to my on-board M.2 drive and SATA Spinning drive. So clearly this process is not bound by disk throughput. CPU utilization only has a slight up tick, so not CPU bound either. A file copy using Windows Explorer, from M.2 drive to SATA SSD, is finished in less than1 sec (better than my reflexes and stop watch can manage). Still close to 1 sec if I use a 2.5GB file. Looks like the blame rests squarely with Adobe on that one. I guess I should not be surprised. It just leaves me wondering what is taking so long with a photoshop file save.

But back to the topic, am I missing something on how those SSD7104/SSD7105 cards could boost performance?