VERY slow raid 5 - why?

Page 2 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.

gse1

Distinguished
Dec 13, 2006
76
0
18,630
8O Doh, sorry. That's what I get for trying to post in these forums and get my work done at the same time. You have no idea how many edits I made to my last post already!

I've corrected that leetle mistake.
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
Oh yes, so he did, my bad. Very interesting this is then! So how could you tell the Raptor was the bottleneck? And did you take into any controller cache? I'm quite interested in this.

Cache was not a factor due to dataset size (a copy of 50GB). The speed of the whole operation was equvalent to reading from the Raptor, and a copy from the first RAID5 array to the second array gave better performance than reading from the Raptor and writeing on the array.
Both arrays consists of 5 500GB 7200rpm SATA harddisks. Can't remember make or model of his controller.

I have serious doubts whether a 5 disk RAID 5 system will beat two disk RAID 0 for write performance. For pete's sake it's all on the original post. This guy has 2 disks in RAID 0 and it was lightning fast and then had 3 disk RAID 5 and it was dog slow. Same hardware, different setup, and it's slower, that's what you'll get with any controller.

As you write he has a 3 disk RAID5 array. I don't dispute that a 3 disk RAID5 is slower than a 2 disk RAID0. Always will be, because in reality there are only data distributed over 2 disks in the RAID5 and there are parity to be calculated and written. A 5 disk array with a good controller shall beat RAID0, if correctly configured.
Anyways there are certainly something terribly wrong with his configuration. If it takes seconds to read ID3 tags from MP3 files in WinAmp, something has gone horribly wrong. The best advice is to reformat and experiment with different parameters, until performance is acceptable - but with 3 disks he won't reach RAID0 levels, and without a dedicated controller, writespeeds won't be optimal.

If you are a heavy Oracle user (with those expensive licenses), with databases which aren't that big, but accessed intensively, then money won't be a problem. If money is no problem, a massive RAID 1+0 with 15k SAS or fibrechannel disks will certainly outperform RAID 5 with a similar number of disks (but sacrificing capasity). But here we are talking a very special situation, and even here it will difficult to recommend the added expense if money has any importance at all.
 

gse1

Distinguished
Dec 13, 2006
76
0
18,630
Cache was not a factor due to dataset size (a copy of 50GB). The speed of the whole operation was equvalent to reading from the Raptor, and a copy from the first RAID5 array to the second array gave better performance than reading from the Raptor and writeing on the array.Both arrays consists of 5 500GB 7200rpm SATA harddisks. Can't remember make or model of his controller.

Ah, I think I might know why that's the case now. Whilst I was browsing through the source material I read that sequential writes can give reasonable performance on a RAID5 system but its random writes that are the problem. So this would make sense.

The following wasn't the original source I remember but it says the same. In fact if you want to know anything about RAID this site is excellent!

http://www.pcguide.com/ref/hdd/perf/raid/levels/singleLevel5-c.html

A 5 disk array with a good controller shall beat RAID0, if correctly configured.

On the same controller RAID 5 with a larger number of disks may beat RAID 0 with a smaller number of disks, e.g. 5 disk RAID 5 over 2 disk RAID 0 (which I still think is disputable but I don't think we'll ever agree on that one) but 5 disk RAID 5 will always lose to 5 disk RAID 0.

Not that I'd ever ever recommend 5 disk RAID 0. 8O

At this point I think we are largely arguing about different things...... and got well away from the original post....

Anyways there are certainly something terribly wrong with his configuration. If it takes seconds to read ID3 tags from MP3 files in WinAmp, something has gone horribly wrong. The best advice is to reformat and experiment with different parameters, until performance is acceptable - but with 3 disks he won't reach RAID0 levels, and without a dedicated controller, writespeeds won't be optimal.

Hmm, well it's most likely a controller configuration or quality issue. If the file system basically works but it is just slow then the basic RAID system is working so either there's some funky settings on the RAID controller or it's just naff.

If you are a heavy Oracle user (with those expensive licenses), with databases which aren't that big, but accessed intensively, then money won't be a problem. If money is no problem, a massive RAID 1+0 with 15k SAS or fibrechannel disks will certainly outperform RAID 5 with a similar number of disks (but sacrificing capasity). But here we are talking a very special situation, and even here it will difficult to recommend the added expense if money has any importance at all.

This is true but my original already-off-topic point wasn't focussed on using databases, I was just using it to demonstrate what I was saying about RAID 5 write performance being bad. At the end of the day, you should never implement RAID 5 if you are focussed on performance, only if you want a large reliable and cost-effective filesystem.
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
Not that I'd ever ever recommend 5 disk RAID 0. 8O
Exactly - RAID 0 is a questionable idea with two discs, but as it goes from questionable to bad, and on to atrocious when adding more disks.

This is true but my original already-off-topic point wasn't focussed on using databases, I was just using it to demonstrate what I was saying about RAID 5 write performance being bad. At the end of the day, you should never implement RAID 5 if you are focussed on performance, only if you want a large reliable and cost-effective filesystem.

Only in very special cases, have you only got one focus. Small highend Oracle databases where money is no object is one such case.
Generally RAID5 performs very well (not best) across a lot of usage scenarios, and starting with a good controller and 5 disks, it is also very performarnce/cost efficient. I am always balancing the requirements when making recommendations, and except for outlaying cases (very small systems or highend databases) RAID5 gives a lot of performance and capasity for the money. And it should certainly always beat single drives......
 

TeraMedia

Distinguished
Jan 26, 2006
904
1
18,990
To the OP: I'm running 4-disk RAID 5 on an Intel MB that has matrix storage functionality. My reads are lightning fast - which yours should be for loading a playlist. My writes are slow because I deliberately disabled the write back cache as I do not have battery backup.

If your RAID 5 array is currently rebuilding (there should be an icon in the system tray near the clock or something to indicate this), then your disk drive system will be slow. Also, if they configured your stripe size wrong, that can slow everything down as well.

The Silicon Image controller is indeed software-only. I use that controller on my MB for a RAID 0 array. It works ok for that, but I wouldn't consider it for RAID 5. The Intel controller (ICH7R) on my MB includes some circuitry to help improve the performance of RAID 5 and reduce the tax on the CPU. And with the writeback cache turned on, I do indeed get decent performance. Toms wrote a review about this a while ago comparing the ICH7R, ICH8R and nVidia 680i southbridges, and found the same.

So, check your MB to see what controllers it has on it. If it's a 955X, 965 or 975X Intel Express chipset, then you should have a controller better suited to RAID 5 already - but you will need to move your drives over to that controller and rebuild everything.

RAID 5 with 3 disks isn't a good idea, as other posters have noted. Consider getting a 4th and making it a RAID 10. It will be much easier to manage, and easier (and faster) to fix if a drive fails.
 

gse1

Distinguished
Dec 13, 2006
76
0
18,630
Exactly - RAID 0 is a questionable idea with two discs, but as it goes from questionable to bad, and on to atrocious when adding more disks.

Don't put words into my mouth. RAID 0 with 2 disks is a very good idea on a performance oriented system. RAID 0 with more disks is a very good idea on a specialised performance system.

Only in very special cases, have you only got one focus. Small highend Oracle databases where money is no object is one such case.

- On filesystems where the focus is performance RAID 0 beats RAID 5.
- On small filesystems where the focus is reliability, RAID 1 beats RAID 5.
- On large filesystems where the focus is reliability, RAID 5 beats RAID 1.
- On large filesystems where the focus is performance and data security is also important, RAID 01/10 beats RAID 5.

There's about a million places on the web that say RAID 5 performance is less than other RAID levels and you accuse me of only having one focus? You seem to think RAID 5 is best for everything.

Generally RAID5 performs very well (not best) across a lot of usage scenarios, and starting with a good controller and 5 disks, it is also very performarnce/cost efficient. I am always balancing the requirements when making recommendations, and except for outlaying cases (very small systems or highend databases) RAID5 gives a lot of performance and capasity for the money. And it should certainly always beat single drives......

Well from enterprise level experience I disagree! RAID 5 is not considered anything more than adequate for performance and should never be the focus for getting a RAID 5 system put into place. RAID 5 is a good system and is very widely used in the industry as it is a good "general purpose" solution. The irony is that I mainly use RAID 5 filesystems at work, even on some database servers, but that's because performance isn't an issue.

I'm bored with arguing about the RAID 5 vs single disk write performance. I did a load of benchmarks on a hardware controller about 5 years ago but I can't find the data.
 

proof

Distinguished
Oct 16, 2006
1,620
0
19,790
I have one 36GB SCA SCSI with a host adapter and two Western Digital Caviar 16MB 250GB. What is the IDEAL setup for performance on these 3 drives? I was thinking a stripe but that would mean the array is only as fast and big as the slowest/smallest drive. Is there a way I can maybe RAID the SATAs and boot from the SCSI? I want pure, no punches pulled, screaming perdformance. (With what I have)
 

gse1

Distinguished
Dec 13, 2006
76
0
18,630
Ask me an easy one why don't you! 8O

You asked for a pure performance answer, it all depends on the capabilities of the RAID controller and what you mean by pure performance

If it's a good controller and you mean file transfer rate as being pure performance then creating a RAID 0 logical across the first 36GB of all 3 disks and a second RAID 0 logical disk across across the remainder of the 2 WD drives should give best performance.

If it's not such a good controller, i.e. it can't do the above configuration or performance across different drive types suffers, or by pure performance you mean random file access times, then you want to put the two WD disks into a RAID 0 array and leave the SCSI disk as a standalone disk. The RAID 0 array will give you best transfer rates and the SCSI disk will give you shorter random access times.
 

proof

Distinguished
Oct 16, 2006
1,620
0
19,790
So I should install the OS on the SCSI and everything else on the two SATAs? And even if the SCSI is not included in the stripe, I can still boot from it and use the SATAs right?
And would a PCI-e SATA RAID card be faster than onbaord SATA RAID? (eVGA T1 680i LGA775)
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
Ok, then lets agree to disagree. But your point-of-view on RAID5 certainly seems strange from my years as a consultant and from what I have learned from for example HP's enterprice storage courses. HP recommends RAID5 for performance filesystems and databases, unless money is no object and the database is relativly small but heavy used.
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
I have one 36GB SCA SCSI with a host adapter and two Western Digital Caviar 16MB 250GB. What is the IDEAL setup for performance on these 3 drives? I want pure, no punches pulled, screaming perdformance. (With what I have)

Well if you don't care much about your data, or you are very good doing backups to DAT, LTO or external harddisk, then you should let the 36GB SCSI drive be system drive (no RAID) and the two WD drives be RAID0.

So I should install the OS on the SCSI and everything else on the two SATAs?

Yes.

And even if the SCSI is not included in the stripe, I can still boot from it and use the SATAs right?

Thats another yes. And when your SATA RAID0 goes poooff, then you can still boot your system. You can even experiment to your hearts content with different RAID stripe sizes, block sizes and other parameters, as you don't need to reinstall your OS every time.

And would a PCI-e SATA RAID card be faster than onbaord SATA RAID? (eVGA T1 680i LGA775)

Depends. The 680i is not the best RAID, but then again, it's not that bad. And RAID0 is NOT very taxing from a controller point of view, so you should be ok with the onboard solution.
 

proof

Distinguished
Oct 16, 2006
1,620
0
19,790
Well I have a 6 port PCi-E SATA RAID Controller. I also have a few more SCSIs. I have 2 15k SCSIs and 2 host adapters. What Should I do? How should I use the 2nd one? Paging file? I have 2gb of RAM so i turned paging off.
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
Well I have a 6 port PCi-E SATA RAID Controller. I also have a few more SCSIs. I have 2 15k SCSIs and 2 host adapters. What Should I do? How should I use the 2nd one? Paging file? I have 2gb of RAM so i turned paging off.

Nice little hardware collection 8)

It's difficult to judge how your PCI-E SATA controller compares in RAID0 to the integrated controller. The advantage of the PCI-E RAID controller is typically in RAID5, but thats not what you are after.
I would:
Install one SCSI host adapter, and use one of the 15k SCSI drives as OS drive without RAID.
Use the other SCSI drive as swap drive without RAID.
Use the two SATA drives in RAID0 on the integrated RAID controller for everything else.
Save the PCI-E controller for later, when you buy those 5 * 1TB Hitachi SATA drives for that nice 4TB RAID5 array, to house your media collection :D

I'm a bit in doubt if you have 3 SCSI drives (1 SCA and 2 68pin SCSI) or only 2. If you got 3, then install both hostadapters and use the SCA as systemdrive, and the two 15k SCSI in RAID0. That would be extremely fast.
 

proof

Distinguished
Oct 16, 2006
1,620
0
19,790
My collection includes a fairly large supply of 15ks and 10ks. Of course I am going to use the 15ks and I only have host adapters though. So one system drive (15k SCSI) one page/swap disk (15k SCSI) and two everything else disks (2x WD Caviar 16MB 250GB SATA). I don't have a SCSI RAID card. (They are all SCA)
 

gse1

Distinguished
Dec 13, 2006
76
0
18,630
So I should install the OS on the SCSI and everything else on the two SATAs?

Well I'd recommend you use the 36Gb disk purely for backup, but if you aren't bothered about that I honestly don't know what would give better performance. The SCSI disk would like give better random access times, meaning accessing lots of small files would be quicker, but the RAID 0 array will give better throughput, meaning accessing large files would be quicker.

And even if the SCSI is not included in the stripe, I can still boot from it and use the SATAs right?

Yes, this should definitely still be possible!

And would a PCI-e SATA RAID card be faster than onbaord SATA RAID? (eVGA T1 680i LGA775)

I don't know. Intel's ICH RAID is fairly mature these days but some other onboard controllers aren't quite as good. Plus it depends one the quality of the PCI-e card. Your best bet is to try and find some benchmarks and look at performance and overhead because most onboard and I suspect some add-on RAID controllers are partially software based.
 

bga

Distinguished
Mar 20, 2006
272
0
18,780
Would a SCSI with a host adapter be better for gaming?

SCSI beats anything not RAID, and SCSI RAID beats SATA RAID. Any 15k SCSI harddisk beats the Raptors, no contest.

Use one 15k drive as systemdrive (lots of small files in Windows), and if you have a SCSI/SCA host adapter supporting RAID0, then take another two 15k SCSI drives and make a RAID0 array for programs and data.

If not, then take a second SCSI drive to be swapdrive, and take the two SATA drives and use motherboard RAID0 to make a data and program drive. Remember to backup.
 

Fedor

Distinguished
Feb 11, 2007
238
0
18,680
Actually storagereview showed that the latest Raptors (ADFD) outperform all the current top 15K SCSI drives in a single-user scenario (including gaming). Something to consider!

Take a look here if you're interested, and the next page beyond that:
http://www.storagereview.com/articles/200601/WD1500ADFD_4.html

Of course the Raptor loses in access times and sequential reads and writes (although is quite close on the latter), but as you can see it beats the SCSI drives hands down in the single-user tests. Since that article the only new SCSI drive worthy of mention was the Cheetah 15K.5 with perpendicular recording, but although it had HUGE read and writes speeds its single-user performance was poorer than the previous generation Fujitsu and Maxtor.
 

Gianti

Distinguished
Feb 23, 2007
35
0
18,530
Would a SCSI with a host adapter be better for gaming?

SCSI would be nice, SAS is much better (Serial Attached SCSI).

And on the discussion here, I have RAID-5 setup with 3 SATA disks running at this time. I've did some benchmarks and compared it to the single SAS drive setup and 2 disk RAID-0 SATA disks. The final results putting RAID-5 with 3 disks very close to the RAID-0 in terms of reads and about 10% slower in writes, single SAS drive lost to both, though it performace was close and impressive. RAIDs were tested on the Intel's ICH-8R and SAS benchmark was ran on the blade server (designing them... so have access to the server hardware).

My view on this, you sacrifise a bit on the write performance with R5 but you get better reliability then R0. If you are going into 4 disk scenarios then the size comes into the picture.

The performance question of the post could be result of the few things:
1) One of the drives is failing
2) Improper RAID set-up (disk mismatch etc)
3) If it is based on software - may lead to 30-40% degradation of performance.
4) RAID table is not set up. There is some time needed to setup and buid parity table.
 

proof

Distinguished
Oct 16, 2006
1,620
0
19,790
I think I will set up the following:
System Disk: 15k 36GB SCSI
Swap Disk: 16k 36GB SCSI
Programs: Western Digital Caviar 16MB 250GB (x2) RAID0 16kb
 

Gianti

Distinguished
Feb 23, 2007
35
0
18,530
Yeah, but name someone who can afford a SAS drive and a controller card...

I can and it wwould be close to free =) I work with them and we get lot's of outdated but still good parts =). So 74G SAS and a nice controller SAS/SATA x10 might end up in trash any time....
 

Gianti

Distinguished
Feb 23, 2007
35
0
18,530
GOOD $HIT MAN! Better guard those cans... I am getting my SCSIs from work. Anyone wanna buy some? Free for me.


I really can't say that I would run SAS at home, it doesn't seem to ba a good option - strains the PCIx bus and puts some heavy load on the system, generally it might end up in a very low performance and the same would go to the SCSI controller. Besides, there always beed issues with OS where it will fail to recognise or use high end server grade controllers.