Question Looking for NVME with great write performance/price - Need Advise!

[ketrab]

Hi,

Working on a multiple build project that will have focus on write performance onto the nvme at gen4 speeds.

Been always the fan of the WD brand (i.e sn770) but with the price point and qty I'm planning to purchase looking for alternative.

I've noticed Team Group MP44L 1TB/2TB variant has a good performance on paper.

Would like to know your opinion on this matter, would appreciate any feedback!

PS. Techpower up also mention this drive stating got a small cache but writes outperforms other vendors?!:

Just like all other modern TLC drives, the MP44L comes with an SLC cache that absorbs incoming writes at high speed, but uses three times the storage to do so. Our testing reveals that the SLC cache is sized at 63 GB, which is surprisingly small for a solid-state drive in 2023. On the other hand, sustained write performance is better than most competitors—we were able to fill the whole 1 TB capacity at 1.45 GB/s, which is a huge improvement over drives such as the WD Blue SN570 (590 MB/s), Samsung 980 (600 MB/s), WD Black SN770 (630 MB/s) and XPG Atom 50 (805 MB/s). So if you plan on writing a lot of data regularly, do prefer the MP44L.

 

[hotaru.hino]

What kind of data will you be writing to it? If it's a bunch of small bits of data, you need something with a lot of IOPS or a high 4K random rating. Note that you'll never hit the advertised maximum write speed in this scenario, as that speed is only attainable if you're writing huge blobs of data at once and as long as the SSD still has space in its SLC cache.

 

[ketrab]

What kind of data will you be writing to it? If it's a bunch of small bits of data, you need something with a lot of IOPS or a high 4K random rating. Note that you'll never hit the advertised maximum write speed in this scenario, as that speed is only attainable if you're writing huge blobs of data at once and as long as the SSD still has space in its SLC cache.

Application will run dd test from time to time. You may expect 6-10 instances of this test running at the same time on single drive.

So this is synthetic 1GB test that will run on interval basis where 1TB disk should run around 4-6 at the same time and 2TB disk 5-10 tests:

dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync; rm -rf sb-io-test

 

[hotaru.hino]

What's the point of running these tests repeatedly?

 

[Maxxify]

Small caches usually relate to high TLC write speeds. That's how it works. If you're looking for the best sustained/steady write speeds from consumer drives, they cap out around 3.8 GB/s in TLC mode but these don't necessarily have the best speed if you write forever. This would be E18/IG5236 + 176L Micron TLC and 2TB of flash. Consistent writes at 1.5-2 GB/s can be found in other drives but they won't hit as high in TLC.

Frankly, you don't want consumer drives for this. You want drives without SLC which is DC/enterprise. They are designed for steady state performance and are more reliable. There aren't too many consumer drives left on the market that are good for these types of workloads (e.g. Gen4+). Of course that's also why consumer drives have lower TBW to prevent that use. Also no capacitors and extra features, but I digress. It is possible to remove SLC caching or make the drive only pSLC with MPTools, and Phison has full-pSLC drives with the E18 on the dock (still), but these are not for the average user.

 

[ketrab]

What's the point of running these tests repeatedly?

Its a benchmark that runs every couple days. Docker Application needs to meet minimum speed of 180MB or otherwise is tagged as failed. Multiple docker apps on single drive thus looking into gen4 to help with that.

 

[ketrab]

Small caches usually relate to high TLC write speeds. That's how it works. If you're looking for the best sustained/steady write speeds from consumer drives, they cap out around 3.8 GB/s in TLC mode but these don't necessarily have the best speed if you write forever. This would be E18/IG5236 + 176L Micron TLC and 2TB of flash. Consistent writes at 1.5-2 GB/s can be found in other drives but they won't hit as high in TLC.

Frankly, you don't want consumer drives for this. You want drives without SLC which is DC/enterprise. They are designed for steady state performance and are more reliable. There aren't too many consumer drives left on the market that are good for these types of workloads (e.g. Gen4+). Of course that's also why consumer drives have lower TBW to prevent that use. Also no capacitors and extra features, but I digress. It is possible to remove SLC caching or make the drive only pSLC with MPTools, and Phison has full-pSLC drives with the E18 on the dock (still), but these are not for the average user.

Any recommendation on the drive in that best sustained/steady write category?! Thank you in advance.

 

[Maxxify]

Any recommendation on the drive in that best sustained/steady write category?! Thank you in advance.

Solidigm P44 Pro (Hynix Platinum P41), Samsung 990 PRO. These have lower TLC speeds than many alternatives and won't write as much through 15 minutes (Tom's Hardware's test), and you even see the Kingston KC3000 beating these two drives by a little on TechPowerUp. Similar E18/IG5236 drives with smaller caches can out-write that by 50% at 15 minutes: MP600 Pro XT, FireCuda 530, Rocket 4 Plus/Plus-G, and a few others (Inland GPP/PP). The reason they out-write is because they hit a maximum TLC speed, but they are less consistent because they will hit folding eventually.

You can't really get around the base TLC speed and further, dynamic SLC shrinks with drive usage (some drives have some or all static and were great drives for your workload - I still use 2xSN750s in RAID-1 to sustained 3GB/s all day; the NAS SN700 is the same drive more or less). However, Gen4 started the trend towards burst performance which makes a lot of drives like the early E16s not good for sustained writes even though people used it for that (the high TBW is misleading). Reviews are misleading because you're not writing 15 min with an empty drive, then letting it recover empty in idle before going again, etc.

Heck, if you look at the 4TB 990 PRO, it underwrites the 4-channel, DRAM-less Lexar NM790 in steady state! (see: TH review) So that's why this requires a lot more explanation than "just buy this drive" and pointing at reviews. Drives just aren't made for tons of writes like that in the consumer space. In my experience, the P44 Pro and 990 PRO have been the most consistent, maybe throw some others in there like the Crucial P5 Plus and WD SN850X, although the first two have the best balance.

One final note: PCPartPicker has engaged in tons of writes to check SLC degradation and found the SN850X to be nearly the best, with high ratings for the E18 (KC3000, FireCuda 530) and P5 Plus and not so great for the Platinum P41 and 990 PRO. There's many reasons for this including the drive shifting SLC algorithms based on workload. Another reason DC/enterprise is ideal. If your intended role will be similar to PCPP's testing, then that leans to the latter two drives (P5 Plus and SN850X, with the SN850X being faster).

 

[ketrab]

Solidigm P44 Pro (Hynix Platinum P41), Samsung 990 PRO. These have lower TLC speeds than many alternatives and won't write as much through 15 minutes (Tom's Hardware's test), and you even see the Kingston KC3000 beating these two drives by a little on TechPowerUp. Similar E18/IG5236 drives with smaller caches can out-write that by 50% at 15 minutes: MP600 Pro XT, FireCuda 530, Rocket 4 Plus/Plus-G, and a few others (Inland GPP/PP). The reason they out-write is because they hit a maximum TLC speed, but they are less consistent because they will hit folding eventually.

You can't really get around the base TLC speed and further, dynamic SLC shrinks with drive usage (some drives have some or all static and were great drives for your workload - I still use 2xSN750s in RAID-1 to sustained 3GB/s all day; the NAS SN700 is the same drive more or less). However, Gen4 started the trend towards burst performance which makes a lot of drives like the early E16s not good for sustained writes even though people used it for that (the high TBW is misleading). Reviews are misleading because you're not writing 15 min with an empty drive, then letting it recover empty in idle before going again, etc.

Heck, if you look at the 4TB 990 PRO, it underwrites the 4-channel, DRAM-less Lexar NM790 in steady state! (see: TH review) So that's why this requires a lot more explanation than "just buy this drive" and pointing at reviews. Drives just aren't made for tons of writes like that in the consumer space. In my experience, the P44 Pro and 990 PRO have been the most consistent, maybe throw some others in there like the Crucial P5 Plus and WD SN850X, although the first two have the best balance.

One final note: PCPartPicker has engaged in tons of writes to check SLC degradation and found the SN850X to be nearly the best, with high ratings for the E18 (KC3000, FireCuda 530) and P5 Plus and not so great for the Platinum P41 and 990 PRO. There's many reasons for this including the drive shifting SLC algorithms based on workload. Another reason DC/enterprise is ideal. If your intended role will be similar to PCPP's testing, then that leans to the latter two drives (P5 Plus and SN850X, with the SN850X being faster).

Thank You so much for dropping knowledge on me here! 990 and/or sn850x would definitely be my primary choice but limited budget need lower tier.

 

[kanewolf]

Application will run dd test from time to time. You may expect 6-10 instances of this test running at the same time on single drive.

So this is synthetic 1GB test that will run on interval basis where 1TB disk should run around 4-6 at the same time and 2TB disk 5-10 tests:

dd if=/dev/zero of=sb-io-test bs=64k count=16k conv=fdatasync; rm -rf sb-io-test

Is the best solution to use up write cycles on your SSD with a periodic dd test?

 

[ketrab]

Is the best solution to use up write cycles on your SSD with a periodic dd test?

unfortunately, what the devs are doing for this specific purpose.

 

[kanewolf]

unfortunately, what the devs are doing for this specific purpose.

That didn't answer the question. Does your VM or docker management system not have tools to identify a hung child ?

 

[Maxxify]

Thank You so much for dropping knowledge on me here! 990 and/or sn850x would definitely be my primary choice but limited budget need lower tier.

Lower tier is also possible, depending on your needed capacity. I think you said 1TB/2TB above. Plenty of 1TB options, the issue at 2TB is to avoid QLC. Right now that would be the WD SN770, Team MP44L, Mushkin Vortex Redline (higher end) or equivalent (Silicon Power XS70, ADATA Premium), A440 Pro (higher end) or equivalent, Team MP44 or equivalent.

 

[ketrab]

Lower tier is also possible, depending on your needed capacity. I think you said 1TB/2TB above. Plenty of 1TB options, the issue at 2TB is to avoid QLC. Right now that would be the WD SN770, Team MP44L, Mushkin Vortex Redline (higher end) or equivalent (Silicon Power XS70, ADATA Premium), A440 Pro (higher end) or equivalent, Team MP44 or equivalent.

You mean to avoid the followings you just mentioned starting from sn770?!

 

[ketrab]

That didn't answer the question. Does your VM or docker management system not have tools to identify a hung child ?

Got nothing to do with vm/docker. Its a quality test (at the vm level) that checks if environment is "acceptable" to host any application (dockers). Something that happens prior any automated deployment. It is what it is, can't change that. Wish there was a better option imho its a total waste but good luck trying convince dev guys

 

[Maxxify]

You mean to avoid the followings you just mentioned starting from sn770?!

You said your budget was too limited for the drives I recommended first. I listed some cheaper options that can in many cases have decent write performance.