Low IOPs using LSI adapter even after firmware update

[dracarys]

Hello,

I'm getting low random read/write IOPs in between 50-60k (sata 2 speeds), as soon as I plug a SSD back into my motherboards SATA 3 ports I get the advertised speeds. However, Sequential read/write speeds are high across the board, only the IOPs are being affected by this adapter. I'm only running JBOD no raid.

Specs:

http://www.ebay.com/itm/New-IT-Mode-LSI-9211-8i-SAS-SATA-8-port-PCI-E-6Gb-s-Controller-Card-/252344098349?hash=item3ac0e15e2d:g:-w4AAOSwGYVXAyCn&rmvSB=true3930k
GA UD5 x79
Samsung EVO 840/850, 256gb and 500gb
Samsung 840 PRO 500gb

I currently have the latest IT firmware installed (2118it.bin / v20.00.00.00) and the newest driver from 2015 (2.0.76.0) opposed to the initial 2009. However, I was told that this is a SAS 2.0 device and I will only get sata 2 speeds. This makes no sense to me, because my sequencial read/writes hover around 550mb, its only the IOPs that are low. Any idea what's going on? My IOPs went from max of 60k to 70k after firmware update, but I should still be getting 90k+

Running out of time to return this thing, my last idea could be that it should be in a PCI 2.0 slot instead of 3.0

Thanks!

 

[marko55]

It is SAS 2 but SAS 2 is the same spec as SATA 3. The controller on the 9211 should be able to provide 290,000 IOPS (per its specs anyway). What make/model fanout cable did you use to connect the drive to the SAS card? Also, anything in between the actual drive and that cable (mobile hotswap drive bays or anything)?

If you're getting the sequential throughput match then you've got the necessary bandwidth so don't sweat your PCIe connectivity.

 

[dracarys]

Hey thanks for quick response. These are the other parts in my configuration:

http://www.newegg.com/Product/Product.aspx?Item=N82E16817994178&cm_re=mb998sp-b-_-17-994-178-_-Product&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-VigLink-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=6146846&SID=ip6ldac2yg000kb500053

http://www.newegg.com/Product/Product.aspx?Item=N82E16816116097&cm_re=3ware_CBL-SFF8087OCF-05M-_-16-116-097-_-Product&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-VigLink-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=6146846&SID=ip6le6hc72000kb500053

I thought it might be my ICY DOCK, but I went directly from the LSI to my SSD outside of the dock, same result

It is SAS 2 but SAS 2 is the same spec as SATA 3. The controller on the 9211 should be able to provide 290,000 IOPS (per its specs anyway). What make/model fanout cable did you use to connect the drive to the SAS card? Also, anything in between the actual drive and that cable (mobile hotswap drive bays or anything)?

If you're getting the sequential throughput match then you've got the necessary bandwidth so don't sweat your PCIe connectivity.

 

[marko55]

I'll tell ya, it probably just is what it is with that HBA. Just like the 9260 doesn't RAID SSDs well at all, this HBA's processor is even weaker than that i believe. The 9266 RAIDs SSDs much better due to its more powerful processor (as an example of difference made by a more powerful ROC). You may need a SAS3 capable HBA from LSI just to get more juice. I'm guessing the 9211 really want designed to handle SSD as much as just handling a whole bunch of spin disks.

 

[dracarys]

I have that notion also, but then other guys on forums are telling me the latest firmware should have fixed my issue, and that they're cards have no issue. No idea. This is the most affordable card at the moment, everything else I've seen from LSI 8ports is expensive as hell. I want to flash my bios, but I know that won't do squat as it's only for if I get an Ivy Bridge CPU.

Any cheaper brands with drivers that aren't just awful? So hard to find a reliable sata controller

http://www.newegg.ca/Product/Product.aspx?Item=N82E16816101792&nm_mc=BAC-GDR-CA-PC&cm_mmc=BAC-GDR-CA-PC-_-dyn-_-Hard+Drive+Controllers+%2F+RAID+Cards-_-N82E16816101792&gclid=CNaBvK_J6s0CFQgoaQodUfgJCw

I'll tell ya, it probably just is what it is with that HBA. Just like the 9260 doesn't RAID SSDs well at all, this HBA's processor is even weaker than that i believe. The 9266 RAIDs SSDs much better due to its more powerful processor (as an example of difference made by a more powerful ROC). You may need a SAS3 capable HBA from LSI just to get more juice. I'm guessing the 9211 really want designed to handle SSD as much as just handling a whole bunch of spin disks.

 

[marko55]

I've actually got a 9211-8i in my machine with 4 spin disks and 4 120GB 850 EVOs running off it. I'm on an OC'd 3770K on Z77 (EVGA Z77 FTW mobo). My card is flashed with IR firmware but same version as yours, and I see the same as you. 65K read IOPS and 60K write, across multiple synthetic benching tools (magician, Crystal, Anvil). Thing is I have NEVER seen actual advertised IOPS from an SSD using these tools, so I just don't sweat. I build a lot of workstations on X99 with the latest and greatest boards, 950 Pros, Intel 750s, etc, and I literally never see the advertised IOPS even in these synthetic benching tools. The 950 Pros come close it seems. The intel 750s don't. They claim over 400K random read and the best I can get synthetically is high 300K.

Right now I have a 512GB 850 Pro for my OS drive on a mobo SATA III port and even Samsung Magician, which always seems to spit back high numbers on performance tests, shows me 77.6K read and 70K write. Even lower with Crystal where I know its doing 4K QD32. Granted its my OS drive so its active but I have nothing else running except this browser right now and the basic background apps, which should be just sitting in memory and not chewing much at my drive. Manufacturer specs on this drive? 100K read and 90K write. yeah.....

What's my point in all this? Don't sweat it! Even at 60K your drive is flying. Your system is freakin fast. These SSDs, even when not at their best are still giving you great performance. You don't have a "problem" per say, and nothing's broken or not working correctly, so its all good.

 

[dracarys]

Your absolutely right, high IOPs are negligible in every day tasks, but I do heavy video rendering and heavy music production (orchestra MIDI), and I need all the juice I can get. Also, when I plug these drives into my onboard SATA 3 ports, I get over 90k IOPs, unfortunately I only have two SATA 3 ports on board, the rest are SATA 2 and bullshit SATA 3 Marvel ports that are advertised as SATA 3 6gb.

It's weird though, that only the IOPS are bottlenecked and not the throughput, which I'm thankful for and may just keep the drive.

Can you recommend any other brand that for sure would not bottleneck and is cheap?

I've actually got a 9211-8i in my machine with 4 spin disks and 4 120GB 850 EVOs running off it. I'm on an OC'd 3770K on Z77 (EVGA Z77 FTW mobo). My card is flashed with IR firmware but same version as yours, and I see the same as you. 65K read IOPS and 60K write, across multiple synthetic benching tools (magician, Crystal, Anvil). Thing is I have NEVER seen actual advertised IOPS from an SSD using these tools, so I just don't sweat. I build a lot of workstations on X99 with the latest and greatest boards, 950 Pros, Intel 750s, etc, and I literally never see the advertised IOPS even in these synthetic benching tools. The 950 Pros come close it seems. The intel 750s don't. They claim over 400K random read and the best I can get synthetically is high 300K.

Right now I have a 512GB 850 Pro for my OS drive on a mobo SATA III port and even Samsung Magician, which always seems to spit back high numbers on performance tests, shows me 77.6K read and 70K write. Even lower with Crystal where I know its doing 4K QD32. Granted its my OS drive so its active but I have nothing else running except this browser right now and the basic background apps, which should be just sitting in memory and not chewing much at my drive. Manufacturer specs on this drive? 100K read and 90K write. yeah.....

What's my point in all this? Don't sweat it! Even at 60K your drive is flying. Your system is freakin fast. These SSDs, even when not at their best are still giving you great performance. You don't have a "problem" per say, and nothing's broken or not working correctly, so its all good.

 

[marko55]

Can't say I'm surprised. Sequential throughput is almost always met as its easy on controllers and disk/flash. IOPS are a very different story. By adding an HBA you're putting a 2nd controller in path between the flash and the system; one on the HBA and the drive's internal controller. Plus going over PCIe, then to the HBA's chip, to a SAS port then converted to SATA.

I only play with LSI. I do have an Areca SAS2 RAID card here I could try but you don't pay me enough to do that kinda work. ;-) You've peaked my curiosity though. If you really want to give it a shot, I'd try a SAS3 HBA either from LSI/Avago, Adaptec or Areca. The SAS3 cards are built for sreaming speed so they have ROCs to support it.

Before worrying about it further though, I'd be benchmarking some of the tasks you're concerned about first. Connect the SSD to your HBA and run some job that takes a while to complete, like at least an hour (if possible). Then connect the SSD to your SATA port and run the exact same task and see what your time difference really is. This will depend completely on how the applications move data on/off the drive. If there's a lot of sequential reads/writes (large files, like over 1MB) then you should be just fine. If its actually thrashing the drive pretty hard with a LOT of quick and TINY read/writes then you should notice a difference.

A lot of people get too hung up on these numbers. Its a spec-driven world and manufacturers take full advantage. Like when they can quote 90,000 IOPS at QD32 but put that drive under heavier load and it SUCKS compared to another SSD that has lower quoted IOPS but performs much better under heavy load. Hell, look at how many people get suckered by cheap SSDs with high spec numbers not realizing its a TLC SSD and not MLC then they freak out when they transfer a 2GB file to it and it slows down to like 100MB/s, and they're flipping out because its supposed to be doing 520 like the spec sheet said. Whoopsie.

 

[dracarys]

Thanks for all that, I ran some tests wit my music software, it's all the same pretty much, although loading certain samples into RAM is quicker now, that's about it as far as I know. Samsung Magician is so random, my Samsung Evo 850 and 840 256gb gets 70k random reads and 60k random writes, versus my Samsung PRO 840 only getting 40k Random writes, and like 300sh sequencial writes, which makes no sense because the PRO is a better drive! All firmware is up to date. Don't get it, I've tried adjusting some settings in RSTe, and power options, doesn't make a difference, pretty sure this LSI card is just cheap as hell, I mean it was only 100 bucks. I'm disappointed because this was a bandage upgrade until I'm ready for a DDR4 rig, but right now my 3930k is only 5-10% worse then the newest broadwell 6 core, what's the point?

The only thing left I could think of is reverting back to an older firmware, probably won't do anything. I'm also too lazy to return this so I'll probably just keep it, I guess it makes my case neat and allows me to bypass shitty marvel controllers all together.

I'm pretty sure I won't see high IOPs unless I get a SAS 3.0 card like you mentioned, oh well, I wish I could at least get a consistent 70k read/write IOP, I'd be happy with that

Can't say I'm surprised. Sequential throughput is almost always met as its easy on controllers and disk/flash. IOPS are a very different story. By adding an HBA you're putting a 2nd controller in path between the flash and the system; one on the HBA and the drive's internal controller. Plus going over PCIe, then to the HBA's chip, to a SAS port then converted to SATA.

I only play with LSI. I do have an Areca SAS2 RAID card here I could try but you don't pay me enough to do that kinda work. ;-) You've peaked my curiosity though. If you really want to give it a shot, I'd try a SAS3 HBA either from LSI/Avago, Adaptec or Areca. The SAS3 cards are built for sreaming speed so they have ROCs to support it.

Before worrying about it further though, I'd be benchmarking some of the tasks you're concerned about first. Connect the SSD to your HBA and run some job that takes a while to complete, like at least an hour (if possible). Then connect the SSD to your SATA port and run the exact same task and see what your time difference really is. This will depend completely on how the applications move data on/off the drive. If there's a lot of sequential reads/writes (large files, like over 1MB) then you should be just fine. If its actually thrashing the drive pretty hard with a LOT of quick and TINY read/writes then you should notice a difference.

A lot of people get too hung up on these numbers. Its a spec-driven world and manufacturers take full advantage. Like when they can quote 90,000 IOPS at QD32 but put that drive under heavier load and it SUCKS compared to another SSD that has lower quoted IOPS but performs much better under heavy load. Hell, look at how many people get suckered by cheap SSDs with high spec numbers not realizing its a TLC SSD and not MLC then they freak out when they transfer a 2GB file to it and it slows down to like 100MB/s, and they're flipping out because its supposed to be doing 520 like the spec sheet said. Whoopsie.