12--16 TB Storage Server - Questions on RAID, hardware, OS

buckcm

Distinguished
Oct 20, 2009
110
0
18,680
Don't read this post. Read below first where it is quoted.

I also posted this thread here: http://www.tomshardware.com/forum/252655-32-storage-server-questions-raid-hardware#t1779771.

This is my first time posting on Tom's. I built my own gaming computer in July of 2008 for my birthday as a high schooler. I've done some smaller projects along the way, and I have been looking at computer parts for 2 years now (started to become thrilled just looking at video cards at TigerDirect) and Tom's since 1 1/2 years ago. I've been reading most of the front page articles for a couple months now and check Tom's everyday.


A friend of mine shoots a lot of video at 1080p and racks up 1 GB for every 1 minute of footage. In just a short hour, he has 60 GB of data. He wants something like 16 GB of storage. I tried to find the storage system he was going to buy; that's not important, but I know it was 16 TB of storage for $2500. I know I can do that for cheaper, so I want to build him a server.

My best stab at this is to buy a motherboard with 10 SATA connections. In it I want to put 10 1.5 TB hard drives for a (theoretical) total of 15 TB which is very close to his want of 16 TB.

Motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813131326

Processor: http://www.newegg.com/Product/Product.aspx?Item=N82E16819115056

RAM: http://www.newegg.com/Product/Product.aspx?Item=N82E16820148163

Hard Drives: I am going to use (8) 2 TB hard drives now.


Case: Antec 1200

Video Card: http://www.newegg.com/Product/Product.aspx?Item=N82E16814127436&cm_re=4350-_-14-127-436-_-Product

Power Supply: http://www.newegg.com/Product/Product.aspx?Item=N82E16817341019&cm_re=ocz_700w-_-17-341-019-_-Product

Disk Drive: I'll just a DVD drive and will connect it via IDE (not very important).


Reasons for the parts:

I don't think this computer needs a crazy processor though the above dual core should be sufficient; furthermore, a C2D (or better) is necessary for any motherboards with 8+ SATA connections.

2 GB of RAM is OK, right? What about 4 GB for an extra $30?

I think that 700w is OK for all the hard drives (I did check Newegg's wattage calculator which is probably a bit pessimistic, though for this system it was under 600w). They should be WD Green or the Samsung 5400 RPM.

The Antec 1200 will house 10 hard drives nicely, don't you think?

The motherboard, again, has 10 SATA connections, gigabit ethernet, and RAID support. My friend needs to access the video files when he edits them on his MacBook Pro. This also has gigabit ethernet which is important for connecting to the sever quickly. Backup is another important thing for all this video. I believe RAID 5 is best in this situation; it will take 15 TB and make 10 TB so that any one disk can fail. This motherboard should be able to take all 10 hard drives and make them into one disk with a RAID 5, right?

Now, that storage center I mentioned that had 16 TB of storage cost $2500. I've seen units which cost just that much with no hard drives included. Even if that's the case, I will be well under $2500 with this 15 TB setup (16 with an external, so add $100 or so).

Cost Analysis:

120 (processor)
180 (motherboard)
~1000 (10 1.5 TB hard drives - notice that deal for the Samsung 1.5 TB hard drive was $95, a deal I found on Slickdeals.net)
40 (2 GB of RAM - add $30 for 4 GB)
75 (700w OCZ power supply before rebate)
35 (4350 before rebate)
140 Antec 1200 (I can save $30 if I get the 1200 with the same Samsung I found a deal on)
$20 for extra SATA cables
100 (external 1 TB HD)

I am getting about $1710. Add $30 for shipping maybe. I don't plan on ordering the hard drives until I find Green 1.5 TB for $100 or less (I already saw one today on Slickdeals so this shouldn't be a problem).

Anyways, that is substantially less than $2500 of the aforementioned storage center... I don't even know if that includes hard drives. Thoughts?

Forgot to ask: Can/should I use Linux for this (to save money)? Should I connect the server with Gigabit ethernet to my friend's MacBook Pro through a Linksys router or the sort? Are 1.5 TB hard drives good for RAID? I found that the Seagates may not be the best but is this true of all 1.5 TB hard drives?


And I just read the sticky...

APPROXIMATE PURCHASE DATE: Uh, as soon I get the go-ahead. Late October, early November 2009

SYSTEM USAGE FROM MOST TO LEAST IMPORTANT: Server

PARTS NOT REQUIRED: (e.g.: keyboard, mouse, monitor, speakers, OS): I'm pretty sure it just needs to hook up to a MacBook.

PREFERRED WEBSITE: I live in the US. Newegg is good though I do check around.

PARTS PREFERENCES:above

OVERCLOCKING: Not important.

MONITOR RESOLUTION: I have a Samsung 23". My friend has 2 24" monitors I believe at 1920x1080. I think his will be like mine. I have two computers hooked up to my monitor, one via VGA and the other DVI (thus the 4350 I chose with both).

ADDITIONAL COMMENTS: above
 
This motherboard should be able to take all 10 hard drives and make them into one disk with a RAID 5, right?
Only if you use software RAID (both Windows Server and Linux support software RAIDs). A separate boot hard disk or RAID should be used. Creating a single RAID5 might not be a good idea. Lots of disks in a single RAID can increase performance, but the risk of losing data if more than one drive fails also are increased. Using a bunch of hard disks that are not designed for RAID use certainly doesn't provide an optimal solution. You can't rely on RAID5 to not lose data. A real backup solution is required or use RAID6 to minimize data loss risks. Can he afford to lose everything if the system fails, is stolen or destroyed?
 

buckcm

Distinguished
Oct 20, 2009
110
0
18,680
What is good RAID software? I think you answered another question. He can just access the files on the Linux box with his Mac?

I definitely don't think he wants to lose all his data. This guy, look here: http://kdka.com/local/Todd.Gallagher.school.2.1143561.html, is doing a book as he goes back to school to answer many questions. There will be a video portion; this person was hired to record much of what happens and edit it all. He definitely can't lose it. How much would you actually get from 16 TB with a RAID 6?

My friend had 3 500 GB hard drives in a raid 5. Two had an error simultaneously, and he lost all of his work.

Btw, is there another good option to back up his files? A RAID just seems the best because it does it for you.

What are the other options to configure the hard drives in RAID 5/6? I know RAID 5 works in groups of 3 at the lowest. I could put 3 groups of 3 with 1 drive as the boot drive or have a group of 4. How creative can I be with the motherboard options? Do I need to get software?
 
Even though it's safer, a RAID6 should also be backed up. The size of a RAID6 is the number of hard disks - 2 * the size of each hard disk, e.g., (10 - 2) * 1.5 TB = 12 TB. For 16 TB, you'd need 13 1.5 TB drives.

My friend had 3 500 GB hard drives in a raid 5. Two had an error simultaneously, and he lost all of his work.
That's my point. While a RAID5 is good, it doesn't necessarily decrease the risk of losing all data. A good RAID5 should have a hot standby hard disk just in case one fails. Two hard disks rarely fail at the same time - the issue is that people often don't replace the failed one immediately.
Btw, is there another good option to back up his files? A RAID just seems the best because it does it for you.
A RAID doesn't backup anything. If it fails, then you've lost everything. A good NAS will allow for snapshots and backups to a tape drive or another RAID.

Windows and Linux support software RAIDs. Why do you need to setup RAIDs in the first place? Storing important data on 2 separate hard disks would be better than using a single RAID. Once a project is complete, then all data can be archived to DVDs and only one copy left on a hard disk.
 

buckcm

Distinguished
Oct 20, 2009
110
0
18,680





OK. I got more details about the whole ordeal. This server needs to connect to a MacPro. Will FreeNAS work with Macs?

I read some about FreeNAS. Can you explain better how I use FreeNAS? It is its own OS? Should I do this in conjunction with a RAID?

I'm also going to use (8) 2 TB hard drives now. The motherboard I chose, the Asus P5Q Premium, has 10 SATA ports but they are separated in 4 and 6 (http://usa.asus.com/product.aspx?P_ID=QPhR6dGjcvkYnazE).

Now I'm wondering how to connect the server to this MacPro. Should I use Firewire?

Here is what I'm competing with: http://www.lacie.com/us/products/product.htm?pid=11261.

I forgot to add: I am going to have (8) 2 TB hard drives as well as a smaller boot drive now.
 
Will FreeNAS work with Macs?
I presume that it works, but you'll have to read the documentation to make sure.

Can you explain better how I use FreeNAS? It is its own OS? Should I do this in conjunction with a RAID?
It has it's own OS and it most likely doesn't support the RAID controllers on your motherboard. FreeNAS can be a software RAID and it doesn't require a powerful CPU (no NAS does). With a software RAID, it doesn't matter if the hard disks are connected to more than one controller. I know that you really want to build a 16 TB RAID (it will be 14 GB max since you lose at least one hard disk for parity), but will you create a single 14 TB volume or will you split the RAID into several volumes?

Now I'm wondering how to connect the server to this MacPro. Should I use Firewire?
It depends on the OS, etc. I'm not familiar with Firewire (I never used it), but it might be easier to use Gigagbit Ethernet. The LaCie NAS behaves like an external hard disk with eSATA and Firewire connectivity. It is 16 TB (2 x 8 TB) only in RAID0; it probably is 12 GB in RAID5 (2 x 6 GB). I agree with sub_mesa, but a backup solution is required. You have a lot more of reading to do.
 

buckcm

Distinguished
Oct 20, 2009
110
0
18,680
Thanks for all the help. I got some help on Youtube. Hopefully I figure this out. I found out about Drobo, but it seems way too slow for video editing. I think FreeNAS is still a good, affordable option.
 

ipahophead

Distinguished
Oct 16, 2009
8
0
18,510
Software RAID really stinks; that what you'll get with your proposed motherboard solution.

Video editing HD will require very high speed connection to the Mac. Think Firewire 800 speeds or iSCSI not file sharing via NAS.

Personally I'd run away unless you can build something for 20% of the cost. If you're comparing a build cost of $1.8K to a purchase cost of $2.5K it is not worth it.

Any enclosure in that price range with 16T will also be using software RAID; don't buy it.

In short, make sure that his Mac supports FireWire 800. Then tell him to buy a hardware-based RAID solution with solid Mac-centric reviews. e.g., LaCie 301435U 8 TB 4big Quadra eSATA/FireWire 800.

http://www.amazon.com/LaCie-301435U-Quadra-FireWire-Firewire/dp/B0028NNIM4/ref=sr_1_6

I repeat: Dear lord, don't build this on software RAID and a network connection. It will NOT perform well enough and the huge pain will never be worth it.
 

buckcm

Distinguished
Oct 20, 2009
110
0
18,680
Thanks. My friend was looking at the 16 TB Lacie Quadra for $3600. I see the 8 TB is $1500. I was definitely speculating, and I thought it might work out. I think I'll let this one go.