Very fast RAID 5

[PerryM]

A guy here in UK has written an article about making a system to record uncompressed HiDef video. This needs about 200 MBytes/s sustained write/read. He is using a Foxconn C51XEM2AA board (i.e. nVidea chipset) with 6x SATA drives configured in RAID 5. My problem is that I cannot find ANY published review of this or other similar systems that give a similar performance. Has he hit upon a pretty unique sweet spot or am I missing something fundamental here?

 

[Madwand]

Here's are some less-than-impressive nVIDIA RAID 5 numbers with a more recent chipset.

http://www.tomshardware.com/2007/01/03/the_southbridge_battle/page17.html

I haven't seen the 590 SLI, but earlier nVIDIA ones also had poor RAID 5 write performance.

So I think there's a mistake.

RAID 0 could do that of course.

Intel's ICH8R RAID 5 has been reported to perform very well, but AFAIK, they're limited to 4 ports for RAID 5 (even when 6 ports are available), so should max somewhere around 180 MB/s at best depending on the drives, etc., generally a little short of the magic 200 MB/s number (but perhaps enough to get by?)

With Intel's Matrix RAID, of course you could do tricks such as configure enough space in RAID 0 for capturing / and other short-term needs, and the rest in RAID 5 for longer-term storage.

Thanks for the note. If you're able to confirm such great numbers with any nVIDIA RAID 5 implementation, I'd love to hear about it.

 

[PerryM]

I'm now told by another video correspondent that these sort of numbers are easily possible on a standard Mac Pro chassis, so he is not surprised by them. There are apparently problems with the nVidia implementation with Intel processors. Quite frankly I'm out of my depth here (with the RAID technology) but:
It is definitely 200MB/s and no, nothing less will suffice!

 

[VTOLfreak]

Get a professional RAID controller. The Areca 1231ML for example.
( http://www.areca.com.tw/products/pcie341.htm )
This thing will pull up to 700MB/sec in RAID 6 and 800MB/sec in RAID 5.
All you have to do is add enough drives.
With 10K RPM drives you should only need 4 drives to get that 200MB/sec.
A good choice would be WD Raptor 150GB WD1500ADFD.
( http://www.wdc.com/en/products/Products.asp?DriveID=189&Language=en )
If you need more capacity you could also use 7200RPM drives but you would need more of them to maintain thesame level of performance.

 

[MU_Engineer]

The RAID 5 on NVIDIA chipsets seems not to be limited to the 6xx and 5xx units nor to the Intel versions. My NForce 4 chipset for AMD also has a crappy RAID 5 throughput on 3 250 GB WD Caviar SE16 drives:

Best sustained reads: 80 MB/sec
Best sustained writes: 19-20 MB/sec

It's not simply a Windows driver issue as I'm not running Windows nor am I using the NVIDIA RAID functionality. I'm using the OS's software RAID functions, which has the chipset treat the drives as separate drives. Apparently the two independent controllers do not like to have cross-traffic between then. I switched to an external SATA controller (HighPoint RocketRAID 2310 PCIe x4 4-port) using the same drives, setup, etc. and here are my numbers:

Best sustained reads: 130 MB/sec
Best ssutained writes: 70 MB/sec

That's about what the disks should be capable of! If you need 200 MB/sec read/write capability, you'll want an external controller anyway. Get a good PCI Express unit instead of using onboard SATA ports. You'll be happier. I use that inexpensive HighPoint controller and purely software RAID, but to tell the truth, software RAID usually outperforms hardware RAID and the cards are much cheaper. I'd suggest the HighPoint 2320 8-port SATA-300 controller on PCIe x4 and probably 6 to 8 decent-sized 7200 rpm drives like the WD Caviar RE2s. That will give you plenty of storage and the speed you need.

 

[Madwand]

I'm now told by another video correspondent that these sort of numbers are easily possible on a standard Mac Pro chassis, so he is not surprised by them. There are apparently problems with the nVidia implementation with Intel processors. Quite frankly I'm out of my depth here (with the RAID technology) but:
It is definitely 200MB/s and no, nothing less will suffice!

OK, I happened to have a system that I could reconfigure to NV RAID 5 to test this out again. I had given this up with negative frustration a long time ago, but was very surprised by the current results. Perhaps the driver's changed, perhaps the OS has changed, perhaps my benchmarking has changed and improved. Whatever the cause, I can definitely confirm that NV RAID 5 at this time can sometimes give very good performance.

With a 3-drive NV RAID 5, 32k stripe size, and Vista (RC1), NTFS 64k clusters, running a mix of 2 300 GB Maxtor DM10/Maxline III drives + 1 320 GB Seagate 7200.10 drive, I get 120-140 MB/s sustained read and write performance -- in the outer sectors, as measured using IOMeter (64k transfers, I/O depth 1, 10 GB test file) and manual file transfers (10 GB test file).

This is pretty much perfect performance, as each drive can do somewhere around 65-75 MB/s max, and with 3-drive RAID 5, we have 2 data drives and one parity drive at all times, so we can at best do 2 times 65-75 MB/s, which is 130-150 MB/s. Take a bit off due to overhead, etc, and 120-140 MB/s are very good numbers, and show that the system is scaling well to drive speed. (So faster drives would likely be even faster.)

With 5-6 drive RAID 5, I'd expect peaks around 4 * 65 MB/s = 260 MB/s, and sustained performance > 200 MB/s for at least the first 100 GB or so. (I've confirmed such numbers with my system using 4 drive RAID 0 using 16k stripes, and am extrapolating to RAID 5 with an addition drive for parity).

I did my tests on an ASUS A8N-VM CSM -- nForce 430.

Here's a sample actual file transfer, from a local add-on RAID controller to the 3-drive NV RAID 5 setup:

D:\tools>xxcopy /y f:\test\test0\10.gb n:\test\test9

XXCOPY == Freeware == Ver 2.93.1 (c)1995-2006 Pixelab, Inc.
...
-------------------------------------------------------------------------------
F:\test\test0\10.gb 10,000,000,000
-------------------------------------------------------------------------------
Directories processed = 1
Total data in bytes = 10,000,000,000
Elapsed time in sec. = 71.19
Action speed (MB/min) = 8429
Files copied = 1

That's 8429 / 60 ~ 140 MB/s actual file transfer performance, writing to the 3-drive NV RAID 5 array.

Thanks again for the posts, I'm very pleasantly surprised by these results. Unfortuntely, I don't yet know why previous results were worse.