Discussion Datacenter SSDs for Home Lab

[bit_user]

With SSD prices near bottoming-out, I got the idea that it's probably a good time to replace some of the older SSDs in various machines I use. I'll be first to admit that none of these is subject to particularly intensive use, but I always like to "over-build" my machines, particularly when it comes to reliability.

I'm curious whether anyone has knowledge of recent data center models that represent a good value, towards the lower-end of the capacity spectrum (i.e. a couple TBs or less). I'm hoping to go with PCIe 4.0 drives (U.2 is fine). Brands I'm partial to are Solidigm (Intel) and Micron, based mostly on prior experience. I definitely have an open mind towards Samsung and Kioxia. I should probably allow SK Hynix, as they're now part of Solidigm.

In terms of budget... I'm looking to stay (hopefully well) under about $500 per drive. In full disclosure, I already bought an Optane P5800X as a boot drive for an upcoming build, but I'm not looking to spend that kind of money on the data drive or other boot drives.

I plan to update this thread, as I shop, but I thought it'd be interesting to solicit recommendations from others and turn this into rather a more social activity than how my online shopping usually plays out.

 

[USAFRet]

Or, you could just do like BackBlaze. Consumer level drives, replace as they die.

And given your stated use....that date of death is likely far in the future.

 

[bit_user]

Or, you could just do like BackBlaze. Consumer level drives, replace as they die.

Thanks, but also I could just overbuild, because it's what I like. Also, I hate wasting time on unreliable hardware. That's why I buy workstation and entry-level server boards, high-quality power supplies, ECC RAM, and run everything at stock speeds with plenty of cooling. A 1500 VA, PFC-friendly UPS with AVR rounds out the setup. On Linux, I go so far as to use filesystems with built-in data checksums, to catch any bit-rot that sneaks through the various levels of hardware protection.

The way some people are with oveclocking and performance tweaking, I get about reliability. That said, I'm firmly in the consumer price tier, so we're not talking about fancy memory mirroring setups or other server-exclusive RAS technologies. But, within my means and practicality, I always go for reliability over performance (or both, to the extent I have a choice). I typically get 10+ years out of my machines and only replace them due to obsolescence.

With deals like a 4 TB Intel P5510 for < $400, like I'm seeing on ebay, enterprise/datacenter SSDs are a very viable option. Samsung PM9A3 is available at comparable prices, if you prefer.

Intel P5510 vs Samsung PM9A3 Enterprise SSD Showdown

Intel P5510 vs Samsung PM9A3: we put these industry heavyweights head-to-head to see which of these enterprise SSDs is the leader.

www.storagereview.com

 

[bit_user]

So, I'm finding the Intel/Solidigm D7-P5520 isn't much more expensive than the P5510. Although both seem to use 144-layer 3D TLC, they claim the P5520 has much better IOPS than the P5510 and lower peak power (but both idle at a rather warm 5W). I'll probably go with this, as it has more features and better performance than Samsung, but can be found much cheaper than the Micron 9400 Pro.

Solidigm P5520 Maintains Lead in Enterprise Workloads

The primary trend that is easy to see across a wide range of tests is that the Solidigm P5520 offers exceptional performance and very low latency across the board.

www.storagereview.com

With performance that strong, one wonders why these aren't more popular in the enthusiast community. Sure, the peak numbers are below the latest PCIe 5.0 drives, but this is a drive you could hammer all-day-long and it'd never even flinch. No hiccups, no stutters, 99.9999% QD1 latency of just 1.1 ms, with average latencies in the range of 80 to 120 microseconds, for most server workloads. There's no way a game is hitting harder than those.

For M.2, Samsung PM9A3 seems the obvious choice. The Intel/Solidigm drives available in M.2 are lower-tier "professional" drives, from what I can tell, but Samsung will sell you a proper datacenter drive in M.2 form factor.

Samsung PM9A3 SSD Review

The Samsung PM9A3 is available in capacities ranging from 960GB to 15.36TB, with a Gen4 interface and the company’s 6th-generation V-NAND.

www.storagereview.com

 

[bit_user]

Okay, so I pulled the trigger on the Samsung PM9A3 M.2 22x110mm 960GB, because one seller has those (new) for $110 (+ free shipping), while the going rate seems more like $130. They're also selling 2 GB and 4 GB models, but those actually cost a little more per GB, and the purpose I have in mind for it doesn't require a ton of capacity, anyhow.

Do note that it's 110 mm long, not the typical 80 mm. Check motherboard (or riser card) compatibility, before you buy. I posted a review, above. Here's the official site:

https://semiconductor.samsung.com/ssd/datacenter-ssd/pm9a3/
​

I'll probably move on the P5520, in a couple more weeks.

 

[sonofjesse]

Do what makes you happy, but consume stuff is pretty decent these days. I have tried quite a few SSD's with great success. YMMV.

 

[bit_user]

Do what makes you happy, but consume stuff is pretty decent these days. I have tried quite a few SSD's with great success.

I have bought probably about a dozen upper-end consumer SSDs, nearly all of them either Intel or Crucial. So far, none has failed me. However, the last one I bought was probably 4 years ago. That means my drives are mostly MLC and TLC (and one SLC gem), and probably all planar.

I also bought one Intel Datacenter drive (PCIe form factor) that's served me well for the past 6 years or so. It seems to me that with datacenter drives selling for this cheap, it's a good chance to get additional performance and robustness for an unusually small premium.

If the pricing weren't so attractive, then I would continue buying upper-tier consumer models. I'm mostly being opportunistic, here. But, I have to say, the U.2 form factor is looking a lot more attractive, in the era of M.2 drives requiring either active cooling or big heatsinks. I have a case with a 140 mm front intake fan that's the perfect place to mount a hot SSD, not my motherboard!

Furthermore, upon reviewing the impressive QoS and low-latency of the Intel/Solidigm P5520, I'd heartily recommend it to gamers.

 

[jeremyj_83]

I have bought probably about a dozen upper-end consumer SSDs, nearly all of them either Intel or Crucial. So far, none has failed me. However, the last one I bought was probably 4 years ago. That means my drives are mostly MLC and TLC (and one SLC gem), and probably all planar.

I also bought one Intel Datacenter drive (PCIe form factor) that's served me well for the past 6 years or so. It seems to me that with datacenter drives selling for this cheap, it's a good chance to get additional performance and robustness for an unusually small premium.

If the pricing weren't so attractive, then I would continue buying upper-tier consumer models. I'm mostly being opportunistic, here. But, I have to say, the U.2 form factor is looking a lot more attractive, in the era of M.2 drives requiring either active cooling or big heatsinks. I have a case with a 140 mm front intake fan that's the perfect place to mount a hot SSD, not my motherboard!

Furthermore, upon reviewing the impressive QoS and low-latency of the Intel/Solidigm P5520, I'd heartily recommend it to gamers.

If you are still looking for drives I would suggest the Micron 7400/7450 Pro. They are lower power drives than the 9400 series so the performance is lower. However, if you aren't running massive DBs you won't notice the difference in the iOPS.

If you stick with PCIe 3 drives, the performance difference will be minimal for you, I would look at the WD Gold or WD Ultrastar DC SN640. They are the same drive just one is more retail focused than the other.

 

[bit_user]

If you are still looking for drives I would suggest the Micron 7400/7450 Pro. They are lower power drives than the 9400 series so the performance is lower. However, if you aren't running massive DBs you won't notice the difference in the iOPS.

Good suggestion. Sadly, the specs sheet doesn't list idle power.

https://media-www.micron.com/-/media/client/global/documents/products/product-flyer/7400_nvme_ssd_product_brief.pdf?la=en&rev=c3d7b6774f6c4b6992da7bc621173904

However, the 7400 Pro is available in M.2 22110 form factor, and for a better price than Samsung's PM9A3.

Micron 7400 PRO - SSD - 3.84 TB - PCIe 4.0 (NVMe) - MTFDKBG3T8TDZ-1AZ1ZABYYR - Solid State Drives - CDW.com

Buy a Micron 7400 PRO - SSD - 3.84 TB - PCIe 4.0 (NVMe) at CDW.com

www.cdw.com

They also sell it in U.3 form factor, which I don't know much about, though it's supposedly 2.5".

If you stick with PCIe 3 drives, the performance difference will be minimal for you, I would look at the WD Gold or WD Ultrastar DC SN640. They are the same drive just one is more retail focused than the other.

Thanks, but I have a few use cases where I actually might notice the performance difference (i.e. copying huge video files). Since I can get enough capacity from the P5520, within my budget, I'll probably stick to that option.

 

[jeremyj_83]

They also sell it in U.3 form factor, which I don't know much about, though it's supposedly 2.5".

U.3 is just the newest version of the U.2 form factor. All U.3 drives are backwards compatible with U.2 connectors.

Thanks, but I have a few use cases where I actually might notice the performance difference (i.e. copying huge video files). Since I can get enough capacity from the P5520, within my budget, I'll probably stick to that option.

If this is over Ethernet or if you have a couple of drive in a RAID I can tell you that there won't be noticeable performance difference. I have a VMware vSAN setup at work with all NVMe storage (WD Gold for capacity and P4800X for cache) over dual port 25GbE. And I am storage network speed limited and not limited by my PCIe 3.0 drives.

 

[bit_user]

U.3 is just the newest version of the U.2 form factor. All U.3 drives are backwards compatible with U.2 connectors.

Cool. Thanks!

If this is over Ethernet or if you have a couple of drive in a RAID I can tell you that there won't be noticeable performance difference.

No, it's for a single-device data volume, in a workstation.

I have a VMware vSAN setup at work with all NVMe storage (WD Gold for capacity and P4800X for cache) over dual port 25GbE. And I am storage network speed limited and not limited by my PCIe 3.0 drives.

Yeah, well a friggin' RAID of NVMe drives is a lot of bandwidth. If you bond dual-25 Gigabit, that's fast enough to keep up with some of the better NVMe 4.0 drives, individually.

 

[jeremyj_83]

Yeah, well a friggin' RAID of NVMe drives is a lot of bandwidth. If you bond dual-25 Gigabit, that's fast enough to keep up with some of the better NVMe 4.0 drives, individually.

That's why I said my storage network is the bottleneck for my vSAN performance. Eventually I hope to move to quad port 25GbE for the storage network. That said even with dual 25GbE I get amazing performance. I have over 100VMs running on that storage and never had a complaint from anyone and getting more Ethernet cards will only help in the long run.

No, it's for a single-device data volume, in a workstation.

Ah that makes it a bit harder. I would probably go with the Micron 7400 Pro just because of the cost. CDW has the 3.84TB M.2's or U.3's for about $250. https://www.cdw.com/search/?key=7400 pro At that price it is hard to complain.

 

[Eximo]

Hmm, hadn't considered looking for 110mm drives. Do have one slot for that...I will make a note of that. Don't need it yet, but soon™

 

[bit_user]

Ah that makes it a bit harder. I would probably go with the Micron 7400 Pro just because of the cost. CDW has the 3.84TB M.2's or U.3's for about $250. https://www.cdw.com/search/?key=7400 pro At that price it is hard to complain.

That's what I linked, but when I can get performance like the Intel/Solidigm P5520, for only an extra $100 or so, it's hard to resist. Sure, the reality is that I probably wouldn't notice the difference, but when I already have an Intel P5800X for the OS/boot drive, it seems a little silly to quibble over $100 for the data volume. I might even put an Intel GPU in that box and make it 100% Intel.

 

[jeremyj_83]

That's what I linked, but when I can get performance like the Intel/Solidigm P5520, for only an extra $100 or so, it's hard to resist. Sure, the reality is that I probably wouldn't notice the difference, but when I already have an Intel P5800X for the OS/boot drive, it seems a little silly to quibble over $100 for the data volume. I might even put an Intel GPU in that box and make it 100% Intel.

Looking at the datasheet for the P5520 it is basically the same as the Micron 7400 Pro.

 

[bit_user]

Looking at the datasheet for the P5520 it is basically the same as the Micron 7400 Pro.

Not even close. I already posted this review of it, above.

​

Solidigm P5520 Maintains Lead in Enterprise Workloads

The primary trend that is easy to see across a wide range of tests is that the Solidigm P5520 offers exceptional performance and very low latency across the board.

www.storagereview.com

​

The D7-P5520 visibly spanks it in every single test, except VDI-LC-Boot, where they're tied for some reason.

That's why we value reviews, and don't simply take manufacturers at their word.

 

[Kamen Rider Blade]

That's why we value reviews, and don't simply take manufacturers at their word.

You should never just take a manufacturer at their word.

The old adage of "Trust, but Verify" are words to live by.

 

[bit_user]

You should never just take a manufacturer at their word.

The old adage of "Trust, but Verify" are words to live by.

I wouldn't even go so far as "Trust". But, the grand irony of that phrase is that truly trusting means not having to verify. So, a more accurate phrasing would be "trust after verifying".
; )

 

[Kamen Rider Blade]

I wouldn't even go so far as "Trust". But, the grand irony of that phrase is that truly trusting means not having to verify. So, a more accurate phrasing would be "trust after verifying".
; )

The Key Part of that "Saying" is that you SHOULD ALWAYS VERIFY. No Matter how trusting you might be.

Everybody's trust levels will vary, obviously; but the VERIFY part is critical.

 

[bit_user]

Well, the 960 GB M.2 Samsung PM9A3 just arrived. Too bad I'm still waiting on the motherboard I wanted to test it in. As it came in "bulk packaging", I'll want to try and verify that it's truly new. This seller did undercut everyone else by $20, but they have excellent feedback ratings.

 

[bit_user]

Update: I bought the the 3.84 TB Solidigm D7-P5520, which just dropped to $268.61!

​

PROVANTAGE: Solidigm SSDPF2KX038T1N1 Solidigm SSD D7-P5520 Series (3.84TB, 2.5 inch PCIE 4.0 X4, 3D4, TLC)

Get Fast Service & Low Prices on SSDPF2KX038T1N1 Solidigm SSD D7-P5520 Series (3.84TB, 2.5 inch PCIE 4.0 X4, 3D4, TLC) and Much More at PROVANTAGE.

www.provantage.com

​

Unlike some deals you might find on ebay, Provantage is an official reseller. This is important, when it comes to things like warranty coverage.

 

[wpu_ccmd]

Update: I bought the the 3.84 TB Solidigm D7-P5520, which just dropped to $268.61!

​

PROVANTAGE: Solidigm SSDPF2KX038T1N1 Solidigm SSD D7-P5520 Series (3.84TB, 2.5 inch PCIE 4.0 X4, 3D4, TLC)

Get Fast Service & Low Prices on SSDPF2KX038T1N1 Solidigm SSD D7-P5520 Series (3.84TB, 2.5 inch PCIE 4.0 X4, 3D4, TLC) and Much More at PROVANTAGE.

www.provantage.com

​

Unlike some deals you might find on ebay, Provantage is an official reseller. This is important, when it comes to things like warranty coverage.

Well, the 960 GB M.2 Samsung PM9A3 just arrived. Too bad I'm still waiting on the motherboard I wanted to test it in. As it came in "bulk packaging", I'll want to try and verify that it's truly new. This seller did undercut everyone else by $20, but they have excellent feedback ratings.

Are you able to provide updates on these?

I went through a similar though process last year wrt the PM1735 HHHL card, but I seem to remember there was some additional factor (other than the length) which made it incompatible with typical desktop mobos