RAID Guru Help Required....

Page 4 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
Just a lookup of the part number returns this info.
Description:
IBM ServeRAID 6M 2CH PCI-x to U320 SCSI w/ 256MB Batt Backed Cache

Overview:
This is an Ultra320 SCSI Controller card made for use with select eServer xSeries models. It has an Ultra320 LVD SCSI interface, transfers data at speeds up to 320MB/s and features a 256MB buffer.It requires item# 457058 02R2068 , Ultra320 Internal 24" LVD SCSI Cable Option.Note: This controller is for external drives only. ServeRAID 7k is required for internal RAID support
Notice the special cable. Also that it is intended for external use only. I think you mentioned the system was using external storage. (dont recall if you had anything internally)

Another thing I havent thought of are the hard drives involved. One hard drive running at Ultra 160 will downgrade the entire array.

http://www-132.ibm.com/webapp/wcs/stores/servlet/ProductDisplay?lang=en_US&catalogId=-840&langId=-1&prmenbr=1&cntrfnbr=1&partNumber=02R0988&storeId=1&cntry=840
 

steelspy

Distinguished
Jan 2, 2007
15
0
18,510
What page did you do the part number lookup on?

If you check the footnote on page

http://www-1.ibm.com/support/docview.wss?uid=psg1MIGR-4PQEWQ

it states you can use the internal or external connector for each channel.

Sounds kinda weird that it would be external only

I also looked up part number 02R2068 and that is an internal cable for the 6M to the SCSI Drive backplane

Can you provide the link to the page you saw?

Thank You
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
I got IO MEter running (someone else was logged on to the box and using it)

RAID 5

32K Sequential Read (1 outstanding IO): 3300 IO/s 104MB/s
32k Sequential Read (16 outstanding IO): 4000 IO/s 126MB/s

32k Sequential Write (1 outstanding IO): 2800 IO/s 84MB/s
32k Sequential Write (16 outstanding IO): 1700 IO/s 53MB/s




On my RAID 1 array (Same controller, different channel):

32K Sequential Read (1 outstanding IO): 1200 IO/s 37MB/s
32k Sequential Read (16 outstanding IO): 2100 IO/s 70MB/s

32k Sequential Write (1 outstanding IO): 1300 IO/s 43MB/s
32k Sequential Write (16 outstanding IO): 1700 IO/s 54MB/s
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
What page did you do the part number lookup on?
If you check the footnote on page
http://www-1.ibm.com/support/docview.wss?uid=psg1MIGR-4PQEWQ
it states you can use the internal or external connector for each channel.
Sounds kinda weird that it would be external only
I also looked up part number 02R2068 and that is an internal cable for the 6M to the SCSI Drive backplane
Can you provide the link to the page you saw?
Thank You
steelspy
Yeah sorry i jumbled those sentences together. The cable is internal. I meant the website said the controller was for external use unless you had the 7k option. Its Sillworks, which is a sales site, they usually are pretty good about these things. But they have been wrong in descriptions before.

Im not sure the link you provided can be held as accurate for whizzard's card as it entails the 39R8816 which replaces the 02R0988 so the details may be different.

Whizzard9992
Didnt you mention two raid1's before?

SupremeLaw
Get my SAS report done?!?!
Go chase an ambulance somewhere.
Or STFU Noob.
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
Didnt you mention two raid1's before?

yeah. They both yielded the same results.

I'm bothered by the results. If I was dealing with a bottleneck, I'd cap out consistently across all arrays. The results for the mirrors seem poor as well, and they don't really illuminate any bottleneck (unless I'm missing something). This means that I'm probably dealing with something related directly to the card, causing it to perform poorly.

(The mirrors have 8k stripes)

I'm running the 7.10 drivers and 7.12 BIOS. I put in a call to have the drivers updated.
 

belvdr

Distinguished
Mar 26, 2006
380
0
18,780
I got IO MEter running (someone else was logged on to the box and using it)

RAID 5

32K Sequential Read (1 outstanding IO): 3300 IO/s 104MB/s
32k Sequential Read (16 outstanding IO): 4000 IO/s 126MB/s

32k Sequential Write (1 outstanding IO): 2800 IO/s 84MB/s
32k Sequential Write (16 outstanding IO): 1700 IO/s 53MB/s




On my RAID 1 array (Same controller, different channel):

32K Sequential Read (1 outstanding IO): 1200 IO/s 37MB/s
32k Sequential Read (16 outstanding IO): 2100 IO/s 70MB/s

32k Sequential Write (1 outstanding IO): 1300 IO/s 43MB/s
32k Sequential Write (16 outstanding IO): 1700 IO/s 54MB/s

Ok, the numbers are looking better, no? I thought you were getting 55-60MB/sec earlier and are now > 100.
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
the mirror results are right on. you should not expect much out of those raid1's. they should be roughly half of a single drives write speed. you could get better performance out of a different stripe size(maybe).

im curious as to how you have two raid1's and a raid5 when there are only two channels available?
I am assuming the two raid1's are being fed off a single channel that is being split by the internal backplane. And the raid5 has the other channel going external all to itself.

Im going to equate this problem most likely to the controller which seems to have been replaced by several newer revisions, probably for very good reason. Or you have a defunct Ultra 160 mixed into your Ultra320 raid5 somewhere.
 

steelspy

Distinguished
Jan 2, 2007
15
0
18,510
I better put down the crack pipe.

I can't hook the backplane to the LSI card. The card is just a SCSI card, not a RAID controller.

I will run IO Meter as well and see what I get. My results have all been from HDTach
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
I better put down the crack pipe.

I can't hook the backplane to the LSI card. The card is just a SCSI card, not a RAID controller.

I will run IO Meter as well and see what I get. My results have all been from HDTach

lol

A tip with IO Meter: When you first run it, the drive will have a strike through it. When you try and run it for the first time, it will say "Preparing drives" Let this run for a minute, then cancel it and restart IO meter (The whole app). When it restarts, the strike through the drive will be gone and you can bench the drive.

It's weird. I don't know enough about the app to explain why it does that.

Actually the mirrors should run at about twice the speed of a single drive; not half. The controller should be able to read from both drives simultaneously, especially if the buffer (block) is larger than the stripe. Writes are generally slightly slower than single-drive speeds, only because of seek times. All of these "shoulds" don't seem to apply to my controller tho. In fact, my reads are slower than my writes on my mirror, which doesn't make sense.
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
Ok, the numbers are looking better, no? I thought you were getting 55-60MB/sec earlier and are now > 100.

Those were different benches. I'll rerun those when I get a chance.


Those 55-60MB/sec were timed with windows explorer and the clock app :) I tried it again with the same results. My access to the machine is limited right now because I don't want to hider efforts currently being put forth by the help desk and IBM.
 

steelspy

Distinguished
Jan 2, 2007
15
0
18,510
I read an interesting tidbit on another forum.

http://forums.2cpu.com/showthread.php?t=57230

The Serveraid 6M is basically an Adapter 3225S card. Its a very fast SCSI raid controller, however, there are a few things to keep in mind.

1) Its a full PCI-X 133 adapter. Designed to work in 100 or 133 mhz slots. It will run slower, but not well
2) Its designed for high end xSeries server chipsets, with 2GB/s links between PCI and Northbridge
3) Its a server adapter, optimized for a server workload (ie: 100% small, random i/o). You're using workstation/consumer benchmarks to test it. Try IOMeter instead. There is a reason we have an 8kb stripe size as default...:)

Quite frankly, while the 6M is an excellent server card, the firmware is NOT optimized for workstation use. I'd recommend one of the LSI Elite series for workstations instead.



I asume you have check that both write cache on the drives and controllers were set to write back. If not, please check.

I had 2x IBM xServer eSeries 346 servers configured in cluster, each had ServeRAID 6M controllers being hooked up to a pair of EXP400 enclosures. RAID5 performance was about 40MB/s, but while in cluster config the firmware would turn write-back cache off automatically. Probably not your case, but worth checking.
Once, I ran a little experiment and disconnected one server from the enclosures, turned ON write-back for drives and 6M and performance soared well above 200MB/s. The 6.11 firmware had some issues with RAID5's and 7.12 was suppose to fix it.


I have no idea how to turn the writeback cache ON for the individual drives. That is, if they mean the individual drives vs the logical drive.

Ain't it nice to see this thread drop from 8 pages to 4?
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
Ain't it nice to see this thread drop from 8 pages to 4?
I was refraining from mentioning that this morning. It's actually quite a nice post now all of a sudden. Im guessing one of the forum gods are having a problem with a 6M somewhere. :p :lol: Its a shame we have such people like supreme to deal with on a daily basis. Some people in this world have completely gone insane.

Good luck with IBM.
Option 1: They spend four days replacing hardware.
Option 2: 5 Minutes after arrival your problem is solved and was so simple you feel like firing yourself.

(crossing finger for option 2)
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
Option 2: 5 Minutes after arrival your problem is solved and was so simple you feel like firing yourself.

lol. That's awesome. Yeah I hope it's something stupid, too. I feel like I'm firing blind in the dark... I hate tech support so I'm glad the help desk is handling it.


@steelspy: Yeah I think I've read that thread like 20 times over :) In particular, this was my favorite part:

...turned ON write-back for drives and 6M and performance soared well above 200MB/s....

At least I know the card is capable of producing those kinds of numbers, and I'm not just living a lie :)

In the ServeRAID directory, there's a RaidMan.bat file that loads the UI. There should be a property there called "Transfer speed" or something like that on the channels. For channel 1, mine is showing "Optimal," which according to the documentation means I'm running U320 (though a better property value would be nice).

Unfortunately I can't check the channel speed on my RAID 5 array because I'm using the EXP400 external enclosure (mid-plane). For some reason that prevents me from checking in the UI. Are you using an external enclosure as well?

I came across an article in IBM's KB that said that they fixed a performance problem in some 6.XX drivers directly associated with the EXP400. I wonder if this problem reappeared with the 7.XX drivers?

I have write-back enabled for all 3 logical arrays. You can change the setting in the BIOS.

@4745454b: Yup :) We had to run a random write script on the database last night, but it only took 2 hours. The last one we ran took 8 :(



Thanks to the admin who cleaned up this thread, and to the person who brought it to their attention.
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
I've been unable to find an article stating at what speed a RAID 5 array should perform.

The help desk actually hasn't called IBM. Instead, they did a file copy on the server and compared it to a file copy on another server. The other server, a 6M card with a RAID 10 array, was able to achieve a whopping 10MB/s. We managed to achieve ~29MB/s.

The conclusion was that "Our server seems to be running pretty fast."

What I need is an article of some sort that demonstrates what the performance should be.

Then again, maybe I'm completely dissillusioned as to how a RAID array should perform? It's just hard for me to believe that my laptop can acheive better performance numbers than my server, and that's OK.
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
LOL you gotta get rid of the help desk first. (not very helpful)

I wanted to find a similar benchmark on one of my servers so i went to the oldest one.

1998-9 Compaq ML370
2x P3 933
1.25G PC133
4x 10,000rpm Ultra 160 SCSI Raid5

Short test: 71 Mb/s average, 90 Mb/s burst.
Long test: 75 Mb/s average, 89.5 Mb/s burst.

You can see why I was wondering if there was an Ultra 160 mixed in.

How bout we trade up, Ill send you my 8-9 yr old server and Ill take that new IBM and make it run like its supposed to. :wink: :p
 

steelspy

Distinguished
Jan 2, 2007
15
0
18,510
I'm getting off topic a little, but is 150MB/sec average read and burst of 190MB/sec a decent result from my external array that is connected to my LSI u320 controller?
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
whats the hard disk setup?

i would guess there is at least 4+ disks to pull 150+ Mb/s

the smart array controller in my compaq server, under all the hp/compaq badging, is an LSI controller.

im getting 150mb/s right on the nose with:
4x 15000rpm Ultra 320 SCSI.
Single channel Smart Array 641

I have several production units with 6 drives in them but i would have to wait until 3am or so before i could run benchmarks on them.
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
I'm getting off topic a little, but is 150MB/sec average read and burst of 190MB/sec a decent result from my external array that is connected to my LSI u320 controller?

It depends on the array, but that looks like what I expect from my array :(

What are you getting on the 6<?

Can yo udo me a HUGE favor and use Robocopy to copy a file to/from the LSI array, and also one that copies a file from the 6M to the LSI? Or did you just move the disks from the LSI to the 6M? :)


Thanks.
 

sandmanwn

Distinguished
Dec 1, 2006
915
0
18,990
hmm i have the option to turn on write-caching for my Ultra 320 but it is not available for my Ultra 160.

this is odd that it does not give you this option. i would guess that it is not seeing the memory on the controller and is therefore not giving you the option to enable caching.

perhaps a driver update will rectify this???
 

Whizzard9992

Distinguished
Jan 18, 2006
1,076
0
19,280
hmm i have the option to turn on write-caching for my Ultra 320 but it is not available for my Ultra 160.

this is odd that it does not give you this option. i would guess that it is not seeing the memory on the controller and is therefore not giving you the option to enable caching.

perhaps a driver update will rectify this???


Yeah I'm going to push to contact IBM directly. I think you're right, that it's a driver issue reporting that "Advanced Performance" is just not "supported" by the device.

I'm really happpy to know that i'm not crazy, though. We have another diesel server we just bought for another project with a RAID 5 SAN; similar configuration, and it gets > 220MB/sec when copying a file. (ServeRAID 8i, Fibre Channel)

I got up to 550MB/s sequential read with DiskBench. I was happy.

Now if I can only fix my volume.
 
I would agree. I have a 3x250GB 7200 rpm RAID 5 setup on Linux md software RAID and have had the experience of goofing around with it before I really had to put real data on it. Here's what I found:

1. Single drive performance
- Maximum sustained read speed: 61 MB/sec
- Maximum sustained write speed: 59 MB/sec

2. Linux md RAID 5, 64K stripe, SATA ports on NVIDIA Nforce 4 MCP (CK804):
- Maximum sustained read speed: ~100 MB/sec
- Maximum sustained write speed: 20 MB/sec

The latter was far less than I expected and after doing some poking around, I saw that NVIDIA's intergrated SATA ports have a real problem with the traffic of RAID 5 writes. So I got an external controller card.

3. Linux md RAID 5, 64K stripe, SATA ports on HighPoint RocketRAID 2310 (PCIe x4, 4 SATA-300 ports)
- Maximum sustained read speed: 120 MB/sec
- Maximum sustained write speed: 67 MB/sec

Fix controller issues and the numbers magically go up a bunch.
 
Mobius is right that RAID generally has little benefit on the average user's desktop. However, he's wrong in saying that a RAID array won't be faster than a single drive as a RAID 0 of very average modern HDDs will have better read and write speeds than the fastest 15,000-rpm SCSI drives. However Mobius must seem to forget that the OP has a database server and that is exactly the kind of thing that RAID is designed for. If he tried to put a critical database on only one HDD, he'd get laughed right out of a job.