Consumer Storage for a Server?

[LukeCWM]

Hey there.

Background: My company is ready to purchase a new server. It will be hosting SQL and Microsoft Dynamics CRM 2011 for 14 employees. We're in the financial industry, so losing data is unacceptable and downtime is costly. But, we're a small company with small means, so if saving money is reasonable, we're all for it. Our current server has a 260 GB RAID5 over three 10k RPM drives.

HP gave me a quote on a server today. When I asked about an SSD, they said their 200 GB Enterprise "Mainstream" SSD costs $2,219! (And the Enterprise "Performance" costs even more!) I asked about 7,200 RPM mechanical drives, and they said a 500 GB drive costs $330 for SATA and $430 for SAS. So much!

I'm finding that enterprise storage is really expensive! When I ask what makes it better, people say reliability and speed. If I need it more reliable, can't I just increase redundancy or add hot-spares all with consumer drives for much cheaper than purchasing enterprise drives? And if it is about speed, can't I just purchase consumer SSDs?

For example, assuming HDD speed is important for a CRM database (another question I have), let's say I buy 4 of the Muskin Chronos Deluxe 240 GB SSDs and 1 WD Red 1 TB HDD. I put three of the SSDs in RAID5 with the resulting array size of 480 GB (an upgrade from 260 GB), and I leave the fourth SSD as a hot spare. Then, once a night, the entire array backs up to the 1 TB drive. A drive fails after a year? No big deal, they are cheap and the array still functions while the hot spare is brought into the array. Suppose another SSD drive fails during the rebuild? That's what the HDD is for. And in the worst case scenario, we still have offsite data backup.

Total cost for the storage I suggested: $770. The result? Blazing SSD speed with zero latency; high quality, long-life toggle-flash memory; redundancy in the array, and extra redundancy just in case. Wouldn't that be a sufficient case for using consumer drives in a business environment?

And total cost if I were to build a three-SSD enterprise RAID5 without a hot spare and without the just-in-case drive: $6,657 for only 400 GB.

 

[kitsunestarwind]

Consumer SSD's are not ideal for reliability in Server Raid environment, In no way shape or form could i recommend that going down that road.

 

[LukeCWM]

Because of reduced speed over time? Lack of garbage clean up? Drive failure? What specifically concerns you?

Thanks for the reply.

 

[HotSpare]

I'm looking to implement same configuration. I have been googling around but not found anyone having done this in production.
Did you implement this? How is it looking? Can you share your experience?

 

[LukeCWM]

I did not implement it yet. Hooray for office bureaucracy. But we're pretty close now. One new server is here, the other is in the mail, and we'll need to order fast storage one way or another within the week.

However, I have done a significant amount of research since I posted this question.

I found one IT professional that recommended consumer SSDs and said he had been installing them for more than a year in servers. But he didn't know they were consumer drives. He told me they were enterprise and then gave me the names and model numbers of consumer drives, haha. And someone else remarked that they wouldn't trust an "IT professional" for anything if he could make a mistake like that and go ahead implementing consumer drives in an enterprise environment.

The difference between consumer and enterprise mechanical storage is that enterprise drives get the most advanced features, are specced for 24x7 up-time, have longer warranties (meaning better quality components, otherwise the manufacturer couldn't afford this), and /significantly/ more research and testing.

What I've also found is that Dell and HP servers by themselves are fairly priced, but they charge through the nose for upgrades, especially RAM and HDDs. So my choice was to buy the servers under warranty from a manufacturer, but light on specs with the intention of upgrading myself from Newegg. Newegg has enterprise Seagate drives for super cheap, and even their SAS 15k rpm drives are reasonably priced. (Even though I specced the servers light, they are coming with a couple HDDs. They are Dell servers of the latest generation, and the HDDs are Seagate, if that makes you feel any better.)

I'm learning more and more from reading that using consumer SSDs in a server environment is a very bad idea. (However, I have a solution that I'll get to below.) The problems are numerous and severe: they aren't built for 24x7 uptime, they have low over-provisioning (affecting speed and endurance), they have shorter warranties (a reflection of the quality), they can fail all at once instead of gradually as many HDDs do, virtually all SSDs slow down terribly once all sectors have been written to, garbage collection processes to fix this have trouble penetrating RAID (and reduce the endurance of the drive when not in RAID), and their write-endurance is quite low. Some consumer SSDs are only good for 20 GB of writes a day for five years. Others are good for 1-2 full-drive-writes a day for five years. But when you factor in any garbage collection or TRIM, that doesn't go very far. Good enterprise drives are good for 10 full-drive-writes a day for five years. Endurance is so important partially because it affects speed long-term, and partially because if you have your HDDs in RAID and their endurance limit is reached, they can all fail at once instead of one at a time like with HDDs. This makes RAID much less useful, unless you introduce fresh drives before others fail in order to have drives of different ages working together.

The nature of SSDs is that a tiny bit of data can't be changed, only written. And you can't delete a tiny bit of data, it has to be a larger chunk. So when your computer wants to delete a file, it marks it for deletion and doesn't actually delete it until it needs the space. And when your computer "updates" a file, it marks it for deletion and writes the updated info in a new place. When the drive runs out of available unwritten space (which happens quite quickly), it has to delete larger chunks that are thinly written and consolidate those to other chunks, which dramatically slows down the speed (the good SSDs slow down to 2x HDD speed, the mediocre ones fall to HDD speed, and the bad ones actually become slower than a HDD!).

The solution to this is Intel's brand new drive: the Intel DC S3700. Newegg currently sells the 200 GB variant for $2.70/GB, although it is supposed to drop to $2.35/GB at some point (much better than the $11/GB HP wanted). But the price isn't the major change. The write endurance is very high, at 3.5 PB over the life of the 200 GB drive under worst-case scenario. And even more unusual is its ability to provide consistent performance: its garbage collection is so efficient that the speed hardly drops, even when the drive is tortured for hours. It remains extremely consistent even then, which we've never seen in an SSD before.

Anandtech's Lal Shimpi said the following: "I view the evolution of "affordable" SSDs as falling across three distinct eras. In the first era we saw most companies focusing on sequential IO performance. These drives gave us better-than-HDD read/write speeds but were often plagued by insane costs or horrible pausing/stuttering due to a lack of focus on random IO. In the second era, most controller vendors woke up to the fact that random IO mattered and built drives to deliver the highest possible IOPS. I believe Intel's SSD DC S3700 marks the beginning of the third era in SSD evolution, with a focus on consistent, predictable performance."

That sums it up perfectly to me. Head over to Anandtech for their in-depth review and awesome graphs that show perfectly what this drive can do that none of the others can.

If you couldn't tell, I'm extremely excited about this new drive, and for the changes it will hopefully bring to all the other SSD manufacturers. However, the jury is still out on whether it is a good fit for my company. Our data is valuable, and even though there are excellent professional reviews for this product, it is still brand new, and I can't find a single review from someone that isn't a journalist. Also, it's quite possible that an office as small as ours simply doesn't generate the IOPS to make use of something faster than fast mechanical. And if management is putting pressure on the budget (which I have a feeling they will), we have to consider that people have been using mechanical storage for databases for decades, that modern 15k rpm HDDs are pretty fast considering that history, and that the 15k drives from Newegg are still about half the price per GB as the S3700 is.

Hopefully I can post within a week what we decided to order. And if the S3700, then hopefully within a month I can post about what the speed is doing for us.

Good luck on your quest, HotSpare. I hope this info is of good use to you, and hopefully others.

 

[LukeCWM]

I just placed an order for 3x 200 GB Intel DC S3700 from J&R through Amazon (they're cheaper than TigerDirect and had them in stock while Newegg didn't). I'll be putting them in RAID 5. I'll post about the results.

 

[Logansacct]

I just placed an order for 3x 200 GB Intel DC S3700 from J&R through Amazon (they're cheaper than TigerDirect and had them in stock while Newegg didn't). I'll be putting them in RAID 5. I'll post about the results.

Did you ever implement this? If so what was the results, benchmarks over time Ect. Also did the server manager give false positive that the drives had failed.

 

[dsteinschneider]

Hi Luke,

I had an unusual scenario where I implemented CRM 2013 for another reseller with 5 users (was able to use the MS Action Pack Workgroup edition). Since we were using Server 2012 R2 Hyper-V we installed three servers on one 8 core SuperMicro with 32GB RAM, one as AD, one running IIS and the other SQL Server. The server has a pair of Samsung 830 256GB SSDs. There's no RAID, instead there is a matching server setup with Hyper-V Replica receiving changes every 30 seconds.

The CRM runs MUCH faster than I've seen it run on traditional 10K SAS in RAID 10. We tested Hyper-V Replica fail over multiple times over a period of 6 months with no errors. I Wouldn't do this in a normal production environment but its working. The owner says they are saving so much on hard drives that he will just clone them to new drives once a year.

All CRM implementations I support are running either MS CRM Online or hosted by a SPLA partner so this scenario won't be coming to one of my clients