WD Red as Desktop Mass Storage?

[GunXpatriot]

I noticed that the Red 6TB just came out. I already was going to order a 4, but regardless...

At the moment, I'm using a 1TB Black, and it's performed well. However, I made two partitions. One 200+ partition for my OS, and the rest for mass storage. This space has pretty much run dry at this point. My OS partition has about 60GB left, and my storage partition is down to under 1GB...

I do video production, especially with game recording. I also dump all of my camera videos, etc on the mass storage partition, and never really sort through it. I mean, it's all organized, but I don't delete much is what I mean. I like to pretty much archive everything I have haha...

So I'm wondering... The setup I would love right now, is a better HDD for my main drive (OS) probably leaning towards a 1TB Hybrid Drive from whoever... And maybe a 4 or 6TB HDD as my secondary/mass storage drive. What are the implications here? I do also have an external 4TB HDD which I think kinda sucks because I have to plug it into an outlet to use it... But my plan was to basically use the internal 4tb as normal, and back it all up onto the external drive, as a "fake raid" so I always have two copies of everything... But alternatively... I guess maybe I could take out the 4TB drive from the external, use it internally, and maybe get a 4 or 6TB for NAS...

What would you guys suggest here? I'd rather have the larger drive internally, but I guess in practicality, one should have an equal or bigger drive externally for backup space... Any ideas? And suppose I wanted to use the Red 6TB internally... That read/bad sector detecting feature certainly couldn't hurt, right? I think the only other issue is that the Red might possibly be slower in performance. But longevity and data protection could trump that, couldn't it? Hmm... :/

 

[Deleted member 217926]

A 250-500GB SSD as your C drive with the OS and programs and a secondary storage drive ( like your 4TB ) would blow your mind with how much it would improve performance.

Do note that the larger HDDs ( 2TB+ ) fail at a much higher rate than smaller drives do. Something to take in mind when backing up data.

 

[joex444]

In so far as capacity and fail rates go, I believe you can model this with 3 points of failure - failure of the SATA board, failure of the spindle motor, and a failure rate for each read/write head. Since capacity is increased by adding more platters, this stresses the motor more and makes it more likely to fail but more importantly the drive is only functional when all the read/write heads have not failed, which leads to an operational probability that scales like 1 - (1-x)^n and when n = 8 (4TB) that's much different than n = 2 (1TB).

 

[geofelt]

Puget systems thinks quite highly of WD red drives.
Read this article:
https://www.pugetsystems.com/labs/articles/Western-Digital-Green-vs-Red-Hard-Drives-602/#Conclusion

For performance on the "C" drive, nothing beats a ssd.
It is 50x faster than the fastest 10k hard drive in random I/O which is what windows does 90% of the time.
It will be 3x faster in sequential.

The hybrid drives do not come with enough nand buffer space to keep anything worthwhile around.
Not a real good option.

And... protect whatever you hold dear with EXTERNAL backup.
In addition to hardware failures, which are rare, you need to protect against viruses, malware, theft, fire and even user mistakes.

 

[Solandri]

I do video production, especially with game recording. I also dump all of my camera videos, etc on the mass storage partition, and never really sort through it. I mean, it's all organized, but I don't delete much is what I mean. I like to pretty much archive everything I have haha...

Okay... You're not gonna like this answer because it's expensive. How important are those videos to you? I mean obviously you care enough about them to keep them. But do you care if one develops a green square for a split second, or the audio develops a burst of static? If you don't care, then you can ignore this post. But if you do care...

Once you get to the point where you have several TB of data, bit rot and bit flips start to become a real problem. That's where a bit in the file changes in a way which isn't detected by the drive's built-in error correction. The chances of that happening in a single file is rather low. But when you have several TB of files, it becomes almost certain to happen over several years.

Bit flips are an even bigger problem. We're constantly being bombarded by cosmic rays from space. They make up about 3%-10% of most people's natural annual radiation dose. They also pass through your computer and occasionally hit it in just the right place to flip a bit. If this happens in the CPU, the program usually experiences a random crash. If it happens in the RAM, your data gets corrupted. Since it's corrupted before it reaches your HDD, the HDD dutifully writes the corrupted data and its error correction assumes the corrupted form of the data is the correct form.

Bit rot can be protected against by newer file systems which can detect it and "heal" files affected by it. Right now the only widely available filesystems which can do this are ZFS and BTRFS. I use ZFS, and every month it automatically scans every file on my file server, checking that its consistent with its checksum and the parity copy. If it detects an error (bit rot), it uses the checksum and parity data to correct the file. Every time it reads a file, it does the same check.

Bit flips require the use of ECC RAM. That's RAM normally used in servers, and has an extra parity bit to go with each byte. While that doesn't completely eliminate bit flips, it greatly reduces the chances of them happening. Basically you need to be hit by two cosmic rays that flip a bit in the byte, and also flip the parity bit in between the time the data is stored in RAM, and when it's read from RAM. ECC RAM used to be very expensive (like 4x the price of regular RAM), but it's dropped down in price to a little more than the price of regular RAM. The real problem is that it's mostly only server motherboards which support it. So getting hardware which can use ECC RAM costs a lot more.

http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/1/

I'd also recommend installing a better file copy handler. I use Teracopy (2.27 - the latest version seems to have more bugs). Windows' regular copy just writes the file to the new drive and forgets about it. If you set the right options in Teracopy, immediately after writing the file it reads it back. It then compares the checksums of the original file with the copied file to make sure they're the same. That way you're assured you have a clean copy (bit flips during the short time the file was in RAM being the only possible error source).

Also, RAID is not a backup. Yes RAID 1 automatically makes two copies of everything. And if you accidentally delete a file, it'll automatically delete its copy at the same time. That means even if you're running RAID, you still need a backup. RAID is for redundancy - if your company would lose millions of dollars an hour if the file server stopped working, then you want to RAID it so a single drive failure will not cost your company millions of dollars.

So if you're serious about reliably keeping copies of your videos, you're going to want to put together a file server. Use a server motherboard which supports ECC RAM. Run either ZFS (FreeNAS supports it) or BTRFS (in beta on many Linux distros). ZFS allows you to arbitrarily group devices together to make the equivalent of RAID if you want redundancy. And you'll need another big drive (or array of drives) to act as a backup for your fileserver. This is going to cost a lot more money than deciding whether to get a 4TB or 6TB HDD or a hybrid drive.