Transfer rates between internal drives REALLY slow

[ansemx324]

Ive tried googling around to find a good answer for this, but nothing is really helping, so maybe you guys could give it a whirl.

I have 2 essentially new WD 2TB drives (WD20EARS). To note, both are error free, I did some really good testing right off to make sure there were no bad sectors or any other smart errors.

Problem: Transferring data between them is REALLY slow (14-15MB/s slow). They are both internal and connected via SATA to a brand new PC I just built (so no old components here).

They both have read speeds of around 100-110 MB/s in hdparm (Ubuntu 10.10), and when I use Esata directly into one of the drives (my Laptop --> Esata to Sata cable --> Drive) I can easily get around 40-50MB/s transferring similar files (Movie VOB files)

Im transferring my ripped DVD Library btw, so they are decently sized VOB files (not lots of small files).

Anyone have any ideas of why this is happening or what I can do to fix it?? Processor is only running at 20-30% also, not limiting anything.

The motherboard is set to Native IDE for the drives (cant switch it to AHCI since Ive installed ubuntu already), but I dont think that should be causing this. Why is it going so slow if they are both SATA II into the mb??

Thanks for the help everyone!

 

[fzabkar]

WD20EARS is an Advanced Format drive. For maximum write performance, it needs to be 4K aligned.

Western Digital supplies an Acronis alignment utility for this purpose:
http://support.wdc.com/product/downloadsw.asp?sid=128

 

[ansemx324]

The drives are both properly aligned; ubuntu 10.10 can handle the new format and aligns them correctly when partitioning (I have checked). Also, if they were misaligned, the individual read speeds would be low as well, but they are above 100MB/s for both.

Only the transfer between the two is bad. Even a USB hard drive to one of the drives transfers faster. Any other ideas?

 

[fzabkar]

I have no experience with Advanced Format drives, but I would think that misalignment would not impact on read performance.

For example, if you needed to fetch only 1 sector out of a block of 8, then you would simply discard the other 7 sectors. OTOH, if you needed to write only 1 sector out of a 4KB block, such as when the clusters are misaligned, then you would read the entire block, insert the desired 512 bytes, recompute the ECC for the 4KB block, and write it to the platters on the next rotation.

 

[Allnsmth]

Hi,
Transfer rate is affected by just about every internal performance factor you can name. The number of platters influences it by changing the mix of head and cylinder switches; actuator design and controller circuitry affect the switch times; media issues and spindle speed influence the all-important underlying media transfer rates. There's probably no other performance specification that is affected by so many different design factors.

 

[Hawkeye22]

Anyone have any ideas of why this is happening or what I can do to fix it?? Processor is only running at 20-30% also, not limiting anything.

20%-30% CPU usage? Is this just during the transfer or do you have other stuff running? That seems aweful high if you have nothing else running.

Try disabling your antivirus. Hopefully it's not trying to scan on copy. Also, make sure the drives are running in DMA mode and not PIO.

 

[ansemx324]

Im running the drives in Ubuntu (linux), so no antivirus running. I dont think anything else besides menial tasks are running, so I think it may be the transfer. I am trying to figure out how to check DMA mode in linux (hdparm isnt really helping, returning weird output rather than telling me), so Ill try to figure out another way.

Thanks for the advice on DMA and PIO though, it took me forever to find that on my own so that is a great suggestion. If I boot into windows and change it, is that in the firmware and will it stay when I boot into Ubuntu? (Ie, is that dependent on OS, or is that something in the drives firmware?)

Thanks for the help!

 

[Hawkeye22]

Ugh, sorry, I didn't see that you are running ubuntu. Your OS controls which mode your drives are running in. I don't think you can change this in the drive's firmware.

I have limited experience with ubuntu, but I think you can check with the blktool. I googled it and this looks relevent.

blktool [options] device [dma off|on]

Ii found it here.

http://manpages.ubuntu.com/manpages/maverick/man8/blktool.8.html

 

[ansemx324]

Thanks for the link!

Thats a great alternative to hdparm...Im wondering if the WD20EARS have issues with DMA, I did a google check and it seems like other people have this issue too. Ill give it a whirl when I get back to the PC and repost to let you all know how it goes (and for future people who might come across this page)

 

[ansemx324]

Ok, so both for "sudo hdparm -d /dev/sda" and "sudo blktool /dev/sda dma on"

I get the output DHIO_SET_DMA: Inappropriate ioctl for device.

Im assuming that this means the "input output control" for this device doesnt support DMA? Thats weird...I thought new hard drives would support this. Am I missing something? Or is that telling me Im typing this wrong?

 

[Hawkeye22]

Sorry, I can't help you there, but even older ATA66 and ATA100 drives support DMA (win98 supported DMA too) so I can say in good faith that all newer drives support DMA.

 

[ansemx324]

Yea, very strange...definitely not fond of this drive right now haha. It seems to be fast if youre transferring stuff with other drives, just not between these too... Thanks for your help! Ill keep working on it and see if I cant come up with something

 

[Hawkeye22]

I don't know if this helps, but this is the only DMA info I can find for ubuntu. Also, it appears that the hdparm and blktool can only be run on a device before it's mounted, thus your error. Maybe you can put the command in a script. The link below shows a CD/DVD player with the settings turned on in the hdparm.conf file.

https://help.ubuntu.com/community/DMA

 

[ansemx324]

That is EXTREMELY helpful to know why that command isnt working...although that makes the command pretty hard to do on the boot drive...

I could boot through a CD version and change the setting, but someone mentioned above that DMA is dependent on the OS, so when I boot from the hard drive, would DMA then be turned off (if it is off in the first place?).
--On a side note, I went into smart and one of the drives shows 1 UDMA CRC error...but only 1 error on one of the drives wouldnt account for the consistent slowness (I tried doing the transfer in multiple steps, and every time it was slow).

In any case, I will unmount the 2nd drive and see what the DMA command says, then if it goes well, Ill give the booting from a CD a shot and see how transfer speeds go

 

[ansemx324]

So as an update, I tried unmounting the second drive (that the OS isnt on) and doing the command and still got the same "inappropriate Ioctl" error.

To see if it was something weird with these wd drives, I tested my DVD drive (ASUS) and it also gave the same error (both mounted and unmounted), so I am confused if hdparm -d even works now.

My thought now is that maybe it is hdparm not playing nice with my motherboard? Is DMA controlled in the motherboard sata controller chipset? This may explain it, I am going to do a little bit of research on it now.

As another note, did some more tests. When copying from one PARTITION to another on the SAME DRIVE, I get around 35-40MB/s. Same drive transfers should be SLOWER than one drive to a different drive (since the hdd isnt going back and forth doing reading then writing). So this has to be some kind of issue with the drive to drive transfer, not the drive's capacity to write.
---Also, the processor was in 95% use when doing that partition to partition transfer, and is about 35-40% in use when doing drive to other drive. This is telling me that DMA isnt working, bc the processor shouldnt be used, right??

If you could can think of anything else, or if you have any idea about the motherboard deal with hdparm, let me know. The board is GA-880GA-UD3H

 

[ansemx324]

Okay, so another update. I have connected each of the 2 drives to my windows laptop via esata. Both drives (on their NTFS partitions) have a writting capacity of at least 65MB/s. I copied 30GB of Dr. Who TV episodes to both drives (seperately), and both were able to take the data from my laptop and write it at 65MB/s.

This means the problem lies in Ubuntu. I may post a question there as well about this. Do you have any other thoughts about the mb or anything else?

 

[Hawkeye22]

All modern motherboards support DMA. The sata drivers in the OS should enable/disable DMA. In windows, CPU utilization during disk transfers is usually around 2% or less. I don't know what is typical in ubuntu but the numbers you stated seem much too high. Sorry, I'm out of thoughts on this one short of trying to reinstall ubuntu.

 

[ansemx324]

Okay, so just to help out anyone who has found this issue, I believe I know what the issue is (sort of). This apparently is an issue with Ubuntu that has been going on for a very long time. Certain users have REALLY slow USB to SATA and SATA to SATA transfer rates, some as low as 1MB/s. Not everyone has this (and I, for example, have a very normal USB transfer rate.

Some solutions seem to be updating the kernal to the latest beta version, changing the mb to using ACHI rather than Native IDE (you can try this without reinstalling), and if that fails, reinstall after switching to ACHI (which still may not work). After that, you may switch to another OS (such as Fedora) to try to fix the issue. I will try to post back based on how I am able to fix this if I am.

I am almost positive mine is a Ubuntu issue since it works in Windows for me. Thank you for all of the help everyone! I really appreciate how great this community was at helping others out with their issues and the brainstorming. Again, the least I feel I can do is update this based on what the end result was so that anyone else who finds it gets some sort of answer.

 

[ansemx324]

Alright, so here is the last updated post before Ill save this thread as solved.

This is a ubuntu problem (which you can sometimes fix, I think I have). The drives are fine, and work with other computers. Ubuntu I guess doesnt always like the Sata controller or SOMETHING like that and it makes transfers REALLY slow. My drives are using the highest UDMA but there was still lots of CPU usage.

How I solved it: I switched my system to use SCSI rather than the native IDE setting as default. This meant redoing my Windows build under SCSI. Ubuntu just automatically picked up on the switch initially and I didnt have to do anything for Ubuntu to boot again, but I reloaded it anyway just to be sure everything was fine. I can get 80-90MB/s with these drives now (AWESOME), BUT THERE IS A CATCH! Sometimes they will fall back down to 15MB/s. This seems to be when transfering between the NTFS partitions. Ext3 partitions work fine with transfers (weird).

Anyway, the fact that I rarely do that many transfers between drives and that it is mostly limited to NTFS is something I can live with. Worst to worst, Ill boot into my windows to do a large transfer if I really have to. Im keeping the NTFS partitions for my movies because that way windows can get at them if need be.

Hope this helps people who are looking for the same issues! I will close this now, but thats a LOT for everyone who helped me, I really appreciate it!

 

[ansemx324]

Best answer selected by ansemx324.

 

[jeffbezos11]

I had the same problem, slow transfer rates between my two internal WD20EFRX 2TB sata drives (~25Mb/s). I hadn't noticed until I bought a Delock 89241 PCIe USB3.0 controller and plugged a USB3.0 ext drive. After I googled pretty much everything and tried H/W settings (AHCI, Bios update, DMA, faulty cables, SATA controller, etc) and tried pretty much every linux (Ubuntu 10.10, Mint 13 Maya mate and live CD of Debian, OpenSuse), the transfer rate was stuck between 25 MB/s and 35 MB/s.

Finally, what did the trick was when I partitioned the drive from NTFS to EXT4 and the tranfer speed climbed immediately to 120 MB/s

Mobo: Asus p5e vm-do, 4 gb ram and Linux Mint 13 Maya (no ms windows on my desktop).

 

[OldGrampa]

Alright, so here is the last updated post before Ill save this thread as solved.


>> How I solved it: I switched my system to use SCSI rather than the native IDE setting as default. <<


When you say "switched to SCSI". Do you mean on your machine Bios, or?
I have the slow copy problem with Ubuntu 13.10 and internal HDD's.
Tx for any comments.