Couple of Raid questions for home file server

jamesho

Distinguished
Apr 28, 2008
14
0
18,510
I'm thinking about building a (hopefully) simple fileserver for home use.

Some questions that I had in mind were:

1. Mobos seem to have a 6x SATA controller built-in using the Intel Matrix technology - is that adequate for hosting a 5 or 6 disk Raid 5 configuration? Or should I be investing in a hardware controller instead? (they tend to be pricy!)

2. Is it possible to expand Raid 5 configurations, and add more hard drives over time? I know it's recommended that you build your Raid 5 configuration with all the disks you intend on using, but it would be nice to expand the capacity as hard drives get cheaper and cheaper...

3. Should all the hard drives of the Raid 5 configuration be of the same type (obviously of the same size) and brand?

Those are my immediate questions, thanks in advance!

James
 

vmardegan

Distinguished
Jul 31, 2008
83
0
18,640
From my knowledge:

Question 1:
If you are only building a home/file server, you shouldn't worry with a dedicated card. Part of your processor will be used to calculate the file allocation in the array, so the higher the number of drives, higher will be the processor utilization. Also, RAID5 uses a lot more the processor than RAID0 or RAID1 for instance, due to the nature of the array. Dual Cores (and especially Quad Cores) have enough room to run the MB array with no major issues.

I am currently running a RAID0 with 3 500 Gb drives and it is blazing fast, with CPU utilization around 1-4%. Same setup on RAID5 was using over 10% and with painful writing speeds, so I switched to RAID0 and now it's a rocket.

Question 2:
AFAIK no. Once you have the array set up, you can hot swap a defective drive, but I believe you can't add more drives to the array. However, I am not 100% sure about this one, so maybe another opinion will be very wise.

Question 3:
AFAIK, not necessarily. Usually it's a good thing to have three equal drives, but this is not mandatory. Keep in mind that the speed will be driven by the slowest drive (faster drives will slow down to match the weakest one) and they need to have (in RAID arrays) is the same capacity, otherwise the bigger drives will be set up with the size of the smaller drive. For instance, two 500 gb and one 320 gb will give you an array of 940 gb and not 1320 gb in RAID0, because it will "decrease" the capacity of the bigger drives to match the smallest one.

Again, this is the best of my knowledge and I'm no expert on the subject. I just have been using RAID for a few years so far and this is what I "learned" by experience.

Maybe if another person that is more qualified than me can give you better advice.

Good luck.
 

einstein4pres

Distinguished
Oct 11, 2007
311
0
18,780
1. Motherboard based RAID should be fine. Since you won't be doing any heavy computing with the processor, the overhead of RAID is fine.

2. I believe this depends on the card. Do some reading about the controller if this matters to you.

3. Generally speaking, you inherit the disadvantages (capacity, speed, latency) of all the drives used. This makes using identical drives more efficient for the most part.
 

chookman

Distinguished
Mar 23, 2007
3,319
0
20,790
1. vmardegan is right with a home windows environment where fast read/writes arent required and you dont need that extra level of features the onboard would be just fine. However DO NOT ASSUME RAID-5 IS A BACKUP no RAID level should ever be used as a backup. If you want to backup data use an external drive or similar.

2. You cant expand your RAID volume with intel onboard. High end dedicated controllers can do this either by adding disks to an array or by linking controller cards and adding that way. This is a time consuming process and will require a full backup before proceding, it can also be limited in implementation (differs between cards and vendors.)

3. You do not have to use the same brand/model/size drive when creating any array, although in general its preferred. As vmardegan stated slower drives and smaller drives are the limitation in heterogeneous arrays
 

enlightenment

Distinguished
Mar 9, 2007
111
0
18,680
It all depends on your needs. A home NAS can be a simple external drive+ethernet or an 18-disk software RAID5 + journaling + encryption array on FreeBSD, such as in my case. The simplest option would be Windows with either onboard RAID or a hardware controller.

However, picking Linux or - my preference - FreeBSD can have certain advantages, especially if you start having more demands on your storage or data-reliability or you have heard about ZFS and want it for yourself. If you want to explore that road, i might be able to help you since there's not alot of info on the net. Even if you never worked with Linux and never heard about FreeBSD, a project known as FreeNAS is dedicated to serving files over a network and provides a very simple web-interface and easy installation.

The advantage of this is that you can use powerful Software RAID engines allowing RAID5 and even RAID6 with very fast write speeds and a much more reliable operation than possible with Intel MatrixRAID (where any due writes saved in RAM are lost when the system doesn't shutdown properly and NTFS' journaling is not able to protect all data in such a case). Normally dedicated Hardware RAID cards use a BBU (Battery Backup Unit) to keep power on the onboard RAM chips to prevent corruption/loss of data. Software implementations available on Linux, FreeBSD (mostly any OS that isn't Windows) allow you to save the need for an expensive controller and run a very modern and safe implementation of RAID5 (or RAID6) on the motherboard's SATA ports. You can also expand by using add-on controllers, which sell for 15 euro at the right place offering 2 ports on PCI-express X1 slot. Never buy a PCI controller, or it will be obsolete already. So even if you have 12 disks connected to 4 different controllers, Software RAID won't care; the controllers have no knowledge about the drives being in a RAID configuration.

That's just an option for you to consider, think about what would best suit your needs and what priorities to set:
- would the benefit of having a real controller outweigh the costs of such a solution?
- how much performance do you need?
- how much capacity do you need?
- how important is it to have expansion support?
- how important is your data, how much of a reliable system you need? (intel ICHxR driver fails here; NTFS is not suited to protect against RAM writeback cache)
- how much time and patience do you have to spend configuring a working storage system? (important if you consider running Linux or FreeBSD)
- will my needs change in the future?

You can go as serious as you want once you explore storage, but for people who want something simple an add-on controller with BBU using Windows makes your life free of hassle and worries. If you like storage alot, like me, and want to explore ways to build a cheap, low-power, ultra-performing, reliable NAS and use modern raid-engines and modern filesystems, a NAS based on Linux or FreeBSD/FreeNAS is highly recommended.

Question 1:
If you are only building a home/file server, you shouldn't worry with a dedicated card. Part of your processor will be used to calculate the file allocation in the array, so the higher the number of drives, higher will be the processor utilization. Also, RAID5 uses a lot more the processor than RAID0 or RAID1 for instance, due to the nature of the array. Dual Cores (and especially Quad Cores) have enough room to run the MB array with no major issues.
The CPU usage does necessarily have to rise as the array gets bigger. Modern drivers (at least geom_raid5 driver under FreeBSD) read only the metadata+datablock and not all the parity when this is not required; meaning the relative CPU usage of the raid5 driver does not grow as more disks are added to the array. It only grows because it is able to move data faster. A good method is to calculate the CPU time per 100MB/s of sequential write throughput, so if you get 50MB/s and the CPU usage is 2% the normalized CPU usage per 100MB/s is 4%. Those values you can compare between RAID-engines.

Also, many drivers, and i think virtually all on Windows, are not able to use more than 1 processor core. But RAID5 performance does not depend on your CPU but on the driver itself. There are dumb drivers (all that onboard fake RAID stuff with the exception of Intel ICHxR RAID5 being mediocre) and smart drivers (such as geom_raid5 on FreeBSD/FreeNAS or ZFS RAID-Z available native on FreeBSD and OpenSolaris).

Thus, you need a smart RAID5 driver in order provide high write speeds; nothing on Windows exists to my knowledge that comes close to the performance offered on FreeBSD.

I am currently running a RAID0 with 3 500 Gb drives and it is blazing fast, with CPU utilization around 1-4%. Same setup on RAID5 was using over 10% and with painful writing speeds, so I switched to RAID0 and now it's a rocket.
If the CPU usage while using RAID5 configuration was only 10%, then that's a clear confirmation the performance is not CPU-bound. Any single core would be fast enough to deliver 200MB/s+ of RAID5 performance, given a good driver.

Question 2:
AFAIK no. Once you have the array set up, you can hot swap a defective drive, but I believe you can't add more drives to the array. However, I am not 100% sure about this one, so maybe another opinion will be very wise.
Expansion support is also still missing for ZFS in RAID-Z configurations. I've written an expansion script for the geom_raid5 driver, but it's still experimental. Expansion on a controller like Areca works great though, just remember you need to grow the partition and filesystem afterwards, too. :)

Question 3:
(faster drives will slow down to match the weakest one)
If you configure an array right, so that 1 I/O requests always lands on one disk, then 3 fast drives + 1 slow drive will still be much faster than 4 slow drives. Tweakers.net also had benchmarks indicating an older WD Raptor with newer WD Raptor in RAID0 config, was still faster than two older raptors.

and they need to have (in RAID arrays) is the same capacity, otherwise the bigger drives will be set up with the size of the smaller drive. For instance, two 500 gb and one 320 gb will give you an array of 940 gb and not 1320 gb in RAID0, because it will "decrease" the capacity of the bigger drives to match the smallest one.
ZFS is a bit smarter here, and can use drives of varying capacity in RAID0 configuration without any capacity being lost. Though it might degrade performance for some parts because it reads from a single disk instead of multiple disks.

Sorry for the long post, once i get started... :(
 

4745454b

Titan
Moderator
I'm thinking about building a (hopefully) simple fileserver for home use.

Ok, so why are we talking RAID if you want a simple file server? If you want a simple file server, what do you think RAID0/1/5 will get you? The only RAID setup I would consider for a simple file server is none. No RAID drivers to get borked causing you to spend a day reinstalling everything, no lost space for more complex RAID setups. A better option would be to just share whatever drives/folders you want, and get a large external USB/Firewire drive for backups. Home file servers won't see enough access to make RAID worthwhile, and the headaches that it can cause aren't worth it.
 

enlightenment

Distinguished
Mar 9, 2007
111
0
18,680
You think a mess of drives connected to various systems, external devices et al isn't causing any headaches? I've seen that first hand, and it isn't pretty. Besides backupping is a mess and you have no flexibility so soon you'll be moving your movies to disks meant for your music etcetc, oh i'm getting flashbacks already.. :p

What he wants may in fact not be that simple. If i understand correctly, he wants to build a new system from scratch, connect 6 drives with some form of redundancy and have future room to expand. That's not a simple setup at all, but may be a simple setup when you confine yourself to the NAS-world, and "home file server" is just that.

So maybe not really that simple but then again everything is relative in life. :)
 

jamesho

Distinguished
Apr 28, 2008
14
0
18,510
Thanks for the information. A fair bit to absorb.

To address some of the (counter) questions...

Ok, so why are we talking RAID if you want a simple file server? If you want a simple file server, what do you think RAID0/1/5 will get you? The only RAID setup I would consider for a simple file server is none.

For me, some data redundancy is better than none. I know that RAID5 does not offer any REAL redundancy especially with its limitation of only allowing one hard drive to fail. Couple that with the fact that the hard drives are housed within the same physical case/location. But it does provide me the greatest amount of (limited) redundancy for the maximum amount of space thereby giving me some peace of mind.

The purpose of this file server is primarily a multimedia server: movies, mp3's, photos and documents - where the vital data would be backed up on an external drive. If I have several terrabytes of music and movies, I'm not expecting to be able to back it up, but atleast I can hope that only one drive would ever fail at a time and therefore I'll be able to keep the array alive for a long time.

To address the suggestion of going FreeBSD...
I must confess that I lied somewhat with my requirements. While the fileserver is intended to be primarily used only as a fileserver, my current athlon 1700xp computer is way to old and needs to be decomissioned. Hence, until I have time to actually build a main computer, I would just use the fileserver as a web browser, media center and to play games whenever I have a chance.

At a first pass, I want to build a RAID5 array with 5 or 6 1TB drives, with a 500gig HD for the OS, applications, and temporary work space. Performance is not a big concern as it is mainly a storage server. Reading mp3's, or even high-definition movies will not require a high performance read from the RAID array. Aside from some wide-sweeping organizational changes in my library of mp3's and what not, writing to the RAID array would be hopefully a one-time thing. And lastly, given that I'm looking at server cases (Lian-Li) that can house 10 internal drive bays, I do want to be able to expand to perhaps another 4 disk array. I think with a potential capacity of 8 TB's or more, I think I should have plenty of space for the next 5 years.

Thanks for the information, and if you have any further suggestions given what I just wrote, please enlighten me!
James

 

chookman

Distinguished
Mar 23, 2007
3,319
0
20,790
Dont think ill dig anymore on the whole RAID and software setup as i think you have enough to think about there.

My main addition would be the cases to look at. I actually have a Lian Li A70 which i find GREAT . however, i was a little upset when they realised the A7010 soon after purchasing mine. The A7010 has hotswappable PCB for 7 drives out of the 10 internals and have better cable management. Those would be the ones id look at. If you can still find an A70 they are cheap because they are EOL
 

enlightenment

Distinguished
Mar 9, 2007
111
0
18,680
Have you thought of what drives to use? If you're going for the energy efficient-route, the WD Green WD10EADS drives are perfect for use in a NAS: they are 1TB big, have a modest price tag, but only consume 2.9W when idling while normal HDDs consume about 7-9W. It is currently the most energy efficient HDD for data storage.

Another thing would be that, if you should ever decide for a NAS based on Linux/BSD, you do not need a seperate system HDD. If you have 6x1TB disks you can just put the operating system on one disk, while using the second partition as data storage. This saves the need for an additional system drive.

Disk1 | FreeBSD/Linux 5G | 995GB Software RAID5
Disk2 | 5GB spare | 995GB Software RAID5
Disk3 | 5GB spare | 995GB Software RAID5
Disk4 | 5GB spare | 995GB Software RAID5
Disk5 | 5GB spare | 995GB Software RAID5
Disk6 | 5GB spare | 995GB Software RAID5

The cool thing about Software RAID, is that it works on partitions.
And building a NAS is actually quite simple and cheap:

Motherboard: ECS A740GM (6x SATA)
EUR 35

Processor: AMD Athlon X2 3800+ EE SFF (35W)
EUR 35

memory: 1GB DDR2/666 CL4
EUR 15

Total: 95 euro, excluding disks, dvd, casing and power supply
Not that expensive really, though these are web-prices in my area. To convert to dollars its pretty much 1 EUR = 1 DOLLAR since we got import tax over here.

About the power supply, if you want to build an efficient server, try the Amacrox 400W Fanless PSU, which has >89% efficiency. Its not that cheap though, i bought it for about 75 euro.