mikenth :
... I have an ancient X58 mb with three M.2 -> PCIe adapters, all populated with Samsung NVMe drives - one 950 for bootimg and two 960's for sw RAID1. The performance is phenomenal -- better on this 7 year old system than what is possible on new DMI-limited mb's. The secret sauce? X58 was a server-class chipset, like X99 / X299 / X399, and had more free PCIe lanes then than Intel-crippled 'consumer' chipsets do now. With any server-class chipset, you can have your cake and eat it too -- get the full x16 performance from one or even two GPUs, and still get the full x4 performance on several other slots populated with M.2 adapters. ...
I've been doing that a lot recently with X79 setups, typically with cheaper SM951/SM961 drives (excellent cache/scratch drives for editing, etc.), but also 950 Pro or whatever for boot (there are a great many older ASUS boards for which modded BIOS files with NVMe boot support is now available). I bought a 1TB SM961 for my Z68/2700K setup, works very well.
There's one other advantage of using M.2 adapters in this way: for older mbds with limited native Intel SATA3 ports, it frees them up for more targeted use, in my case a port is linked to a front-bay hot swap so I can do live C-drive backups or access other drives for whatever reason (3rd party SATA3 controllers generally suck).
A caveat though, albeit perhaps a minor issue: on some older mbds with lots of PCIe slots (mainly X58 and X79), some slots are routed via PCIe switches, which can add a little bit of latency. There's usually at least one x4 or x8 slot though which goes straight to the CPU, as can been seen for example in the following diagram of the ASUS P9X79-E WS (in a 4960X system I recently built, the directly connected x8 slot holds a 512GB SM961, which belts along at almost 3.5GB/sec):
https://images.anandtech.com/doci/7613/Chipset%20Diagram.png
In this case, as mikenth says, one can have several GPUs aswell as a crazy fast NVMe as boot or some other purpose. My 4960X system has an SM951 256GB for the C-drive (only cost 65 UKP) and two 780 Ti GPUs for CUDA which both run at x16.
There's even a P55 board one can do something like this with (not quite to the same extent, but still surprisingly potent), namely the ASUS P7P55 WS Supercomputer, which via two PLX switches supports x8/x8/x8/x8. I have two of these P7P55 WS boards, great for CUDA crunching; I used one of them with three 980s SLI to bag all of the 3DMark P55 records (except for the DX12/VR stuff, as I'm only using WIn7).
There's a lot of life in older tech where one has plenty of PCIe lanes/slots to throw around. I have an Asrock X58 Extreme6 I plan on experimenting with at some point, not gotten round to it yet.
Oh and btw, X79 boards usually support the dirt cheap 10-core XEON E5-2680 v2 (mine cost 165 UKP), a great way to get native PCIe 3.0 and a good performer for threaded tasks (scores 15.45 for CB 11.5 and 1389 for CB R15).
This of course is why Intel and mbd vendors do not want older products to support bootable NVMe, it would extent their life, which means fewer people upgrading. Thank grud for BIOS mods.
Lastly, the 950 Pro is a wonderful drive because it has its own boot ROM, so it can be used even on older mbds that cannot be flashed with a modded BIOS for bootable NVMe.
Ian.