Large raid, Slow speeds

[warlock257]

Hi

I just put together a new computer for storing our digital footage at the office. I set it up as a Raid 5, because I wanted maximum space, but wanted to be safe if one of the drives died. We cannot lose a clients footage, or we'd lose big money.

I chose a ASUS P8P67 B3 Intel P67 motherboard, because it has 8 sata ports. I chose Seagate 2TB, 5900RPM, 64MB, SATA-6G hard drives for the raid. There are 6, 2TB drives, for a total of 12 TB, minus what it needs for the Raid configuration.

Anyway, I'm getting really slow transfer speeds when copying footage to my raid. Its only about 6-8 mbps. This is really bad because it will take all day to copy over 1 project (around 100 gb). This is from one sata drive to another too.

I'm wondering about the stripe size. When I created the raid set, it recommended 64 kb in the utility. Should I set it higher?

Thanks

 

[Chainzsaw]

First off not too sure why you chose such a slow HDD for your RAID setup. That's definately going to cause it to be slower.

Also, did you already copy over the information? Normally if you change the block size to say 128KB you might have to re-do the RAID. It should increase performance, since 64KB is a bit small and requires more CPU power to do 64KB chunks vs 128KB chunks.

Also how exactly are you transferring files? Are you going from 1 HDD to your RAID drives? What is the specification of the HDD you are transferring from?

One last note, built-in RAID on motherboards are not very powerful. You will not get the maximum benefit of RAID without a dedicated RAID controller.

Also one last thing. The speed of the transfer depends also on the size of the files you are transferring. For example - if you have 1000's of >1MB files - that will take forever as the HDD's have to read each file and copy it.

 

[sminlal]

RAID-5 has the very worst write performance of all RAID organizations, especially if you don't have a smart hardware controller with plenty of RAM for write caching. If you're using motherboard RAID then your best option is to switch to a decent hardware card.

And if these files are that critical to you, you're also backing them up to external media, right? Of all the myriad ways to loose data, RAID protects you ONLY from disk failure, and RAID-5 isn't even the best choice for that.

 

[warlock257]

First off not too sure why you chose such a slow HDD for your RAID setup. That's definately going to cause it to be slower.

Also, did you already copy over the information? Normally if you change the block size to say 128KB you might have to re-do the RAID. It should increase performance, since 64KB is a bit small and requires more CPU power to do 64KB chunks vs 128KB chunks.

Also how exactly are you transferring files? Are you going from 1 HDD to your RAID drives? What is the specification of the HDD you are transferring from?

One last note, built-in RAID on motherboards are not very powerful. You will not get the maximum benefit of RAID without a dedicated RAID controller.

Also one last thing. The speed of the transfer depends also on the size of the files you are transferring. For example - if you have 1000's of >1MB files - that will take forever as the HDD's have to read each file and copy it.

I chose that hard drive because it was cheap, basically. They were only $75 each for 2 TB. I already went over budget on this.

I did copy stuff over, but I can start again, if I have to. So changing it to 128 kb will help...okay.

Today I was transferring files from another sata drive, but normally we'll be copying footage from flash cards through USB 2.0. The files will generally be quite large.

It'd be tough to get a separate RAID card. I'd need at least 6 sata ports. As I said, the budget is tight on this.

The media we have already exist on other drives, but I don't know for how long. Was RAID 5 a bad choice? From what I read, it was the best for fault tolerance and the most storage space. If you have a better idea, I'm open to it.

I'll add that I just did a test and copied some videos via USB 2.0 to the operating system drive, which is outside the RAID, and was getting a solid 30mbps, which is totally fine.

Thanks for the replies.

 

[warlock257]

So, if there a better choice than Raid 5 for large video files?

 

[sminlal]

So, if there a better choice than Raid 5 for large video files?

There's a better choice than RAID-5 if write performance is important - you can use RAID-1 or RAID 1+0. But if you just have to copy the files to the RAID volume once and from that point on they'll only or mostly be read, then don't worry about the poor write performance. The read performance of RAID-5 is perfectly fine (as long as the volume isn't degraded due to a drive failure).

 

[leandrodafontoura]

Go RAID 0, an create a backup solutuion

check WD my passport

 

[4745454b]

Why did you not go SSD?

6 * $79 = $475. (rounded) I bet you could get at least two if not 3 120GB+ drives for that much. If the files are already somewhere else and you just need to work on them, a couple of 120GB+ SSDs are fine for that.

 

[warlock257]

Why did you not go SSD?

6 * $79 = $475. (rounded) I bet you could get at least two if not 3 120GB+ drives for that much. If the files are already somewhere else and you just need to work on them, a couple of 120GB+ SSDs are fine for that.

I need size more than speed. No one is actually editing on this computer, its just for storage.


Thanks everyone for your input

[Update]
Increasing the stripe size to 128 kb helped. Its more like 17 - 20 kbps now.

 

[warlock257]

Best answer selected by warlock257.