Question Frquently need to copy 70 gb from one HDD to Another. It takes about 2 hours. What should I do to speed the progress?

Feb 18, 2019
57
3
45
2
Gygabite Ep41UD3l
Core 2 Quad 2.6Ghz
4 Mb Ram
7500 rpm WD HDD (System C)
USB My Passport WD Drive (External)

70 Gb Data Transfer time is over 2 hours. Why is that so? How can I speed up the system?
 
Feb 18, 2019
57
3
45
2
I use simply copy/paste function, no special progs. I'm copying thousands of small files twice a week. I need to do this in archival purposes, as a part of a big data base of clients. Among these files the bulk is txt and docs, other 30% small programs, other material is simple client photoes. All amount is about 70Gb of info. Need to speed up.
 
Feb 18, 2019
57
3
45
2
In them days I Used Blu Ray rewritable disc in archival purposes and it seems to me it took me less time to burn a disc (previously erased), then now throw almost the same amount of info from one HDD to another... The biggest BD RE I have is 50Gb doublelayer RW, but Recently the average info is 70 Gb and I started to use external HDD for the same purpose... But 2 hours is killing me....
 

USAFRet

Titan
Moderator
Mar 16, 2013
114,571
2,116
157,240
18,634
Reactions: Garry423
Feb 18, 2019
57
3
45
2
The problem is that I need to do it during daytime and concerning better hardware you mentioned in a link, I tried to use something like this to obtain usb 3.0 but the only thing I got after Windows is loaded with such kind of card was the sign - This device can perform faster, so...
 
Nov 3, 2019
12
2
15
0
...
USB My Passport WD Drive (External)

70 Gb Data Transfer time is over 2 hours. Why is that so? How can I speed up the system?
I take it you need to use the computer meanwhile, and must power down (why ?) for night. Another good option mentioned is using one of the programs - BUT deliberately make copying slower so you can use computer well while it copies. But best option would still be a copying manager that does it at night.

I think most of the time goes to create each file, years ago I used rar program (REMEMBER to specify no compression or it'll be slow) so they'll be one large file, then move that large file to storage HDD. You can also specify file sizes so it automatically makes backup1.rar 5Gb, backup2.rar 5Gb etc... you'll end up like only 14 files.
 
Reactions: Garry423
Reading from slow drives and writing to slow drives rivals watching paint dry...in the winter....!

Spend ~$200, get a pair of 1 TB Crucial MX500 drives, one for your source, another in a USB 3.0 adapter at least. Consider getting an Icy Dock5.25" front bay or two (one for work, one for home) that holds 2 or 4 of the 2.5" drives...you can copy quickly at work to get the heck out, take the drive with you, and copy to slower media later at your leisure while you watch Star Trek TNG reruns...

With both fast source and destination media, your times should be trimmed considerably...from 2 hours down to under 30 min, I'd guess/hope...
 
Reactions: Garry423
Nov 3, 2019
12
2
15
0
A) Also, someone more knowledgeable, is it too much of a hassle to use raid 1 or somesuch (with 3 HDD's) ? Two HDD's at work at all times (two mirror HDD's), one at home as backup of the data. (Raid 1 keeps two identical copies I recall).

So at backup-day, take one HDD from work to home (new backup copy), next morning you're in trouble at work: comes when you bring ex-backup HDD from home to be overwritten. On plus side the data is always in two places.

Disclaimer: not a clue about raid... Probably much slower to rewrite that data over backup than mdd1963's solution, but due to computers age I think main issue is about 1) money or 2) program(s) of industrial nature whose lifespan/something would require lots of new programming done.

B) Repeat: please do check rar (or some other program that does one file but doesn't compress your backup data), I'd like to hear if it's faster - I bet it will be, since you won't be creating complex file structures to the backup-copy, just few files. Also, rar(or whatever) can "update" the backup w/o touching files IF the files haven't changed. So maybe don't need to copy whole 70Gb, but only files that have changed meanwhile ?

C) Edit:
Did a small RAR test, so 600+ files in software encrypted drive, windows 7, also old computer but usb 3.0. Times:
1: 7 min 30 sec = Plain Copy+Paste the directory to usb stick,
2: 5 min 15 sec (+1 min 30sec used in RAR) = Rar'd that directory to same "at work" HDD first (1.5 minutes), then Copy+Paste that one rar file to the usb stick: 5 min 15 sec (computer slowed a bit while creating rar, was doing other stuff though - using ctrl+alt+del one could put the rar program to one core while doing other stuff with other cores ?).
2.1: Added (changed) the item I was rar'ing (I already had the rar in encrypted HDD, selected add/write over changed or added files), added 2Gb stuff to original directory. Didn't really time, but was rather fast.

I expect with 70Gb lots files/directories RAR'ing w/o compressing file copy to be much better improvement than my about 450 to 315 seconds timing, note I was copying only 615 files against copying 1 file. Also, if you can keep that backup70Gb.rar-file at (work computer) and add/change only changed/added files that part shouldn't take too long. Used: winrar with no compression, hardware has dual-channel 2x4Gb memory, software encrypted 1Tb drive, usb3.0, 4-core 3.0GHz'ish machine. Windows 7 in bit faster drive (not encrypted) alone (none SSD's, well, usb drive but meh). Just bought a "slow" ssd so moving to win10...
 
Last edited:
Feb 18, 2019
57
3
45
2
A) Also, someone more knowledgeable, is it too much of a hassle to use raid 1 or somesuch (with 3 HDD's) ? Two HDD's at work at all times (two mirror HDD's), one at home as backup of the data. (Raid 1 keeps two identical copies I recall).

So at backup-day, take one HDD from work to home (new backup copy), next morning you're in trouble at work: comes when you bring ex-backup HDD from home to be overwritten. On plus side the data is always in two places.

Disclaimer: not a clue about raid... Probably much slower to rewrite that data over backup than mdd1963's solution, but due to computers age I think main issue is about 1) money or 2) program(s) of industrial nature whose lifespan/something would require lots of new programming done.

B) Repeat: please do check rar (or some other program that does one file but doesn't compress your backup data), I'd like to hear if it's faster - I bet it will be, since you won't be creating complex file structures to the backup-copy, just few files. Also, rar(or whatever) can "update" the backup w/o touching files IF the files haven't changed. So maybe don't need to copy whole 70Gb, but only files that have changed meanwhile ?

C) Edit:
Did a small RAR test, so 600+ files in software encrypted drive, windows 7, also old computer but usb 3.0. Times:
1: 7 min 30 sec = Plain Copy+Paste the directory to usb stick,
2: 5 min 15 sec (+1 min 30sec used in RAR) = Rar'd that directory to same "at work" HDD first (1.5 minutes), then Copy+Paste that one rar file to the usb stick: 5 min 15 sec (computer slowed a bit while creating rar, was doing other stuff though - using ctrl+alt+del one could put the rar program to one core while doing other stuff with other cores ?).
2.1: Added (changed) the item I was rar'ing (I already had the rar in encrypted HDD, selected add/write over changed or added files), added 2Gb stuff to original directory. Didn't really time, but was rather fast.

I expect with 70Gb lots files/directories RAR'ing w/o compressing file copy to be much better improvement than my about 450 to 315 seconds timing, note I was copying only 615 files against copying 1 file. Also, if you can keep that backup70Gb.rar-file at (work computer) and add/change only changed/added files that part shouldn't take too long. Used: winrar with no compression, hardware has dual-channel 2x4Gb memory, software encrypted 1Tb drive, usb3.0, 4-core 3.0GHz'ish machine. Windows 7 in bit faster drive (not encrypted) alone (none SSD's, well, usb drive but meh). Just bought a "slow" ssd so moving to win10...
Yes, thank you. Archiver is really an option for me. Thank you. It's really a kinda solvation for my prob. Now do it like you said. One big 7Zip (archiver) file without compression.
 
Feb 18, 2019
57
3
45
2
RAID 1 is not the solution for this. Not even a little bit.

Please describe the actual chain of events. In as much detail as you can.
We can then devise a solution that works with your ancient hardware.
Actual Chain of Events...Hmm.
  1. Come to the working place at 8.00 (the hardest))))
  2. Actual Job.
  3. Around 14.00 I'm staring to copy database from Internal HDD (Sata2) to External HDD (USB 2.0) Now I'm converting info to one big 7Zip file without the compression and off I go for smoke and coffee for an 1 hour and a half))) Still not using any special programs and I've got no Xperience with any. Looking for a good program, but training and working time limits my search. Advice for any program will be of much value. Still using copypaste... The PC is not usable during copying because of much load and during archiving process I forced to use My ASUS EEEPc born in 2005 and it's only for working 'cause it's really a piece of junk. Main problem is money. I'll afford myself SSD a month later, it'll really improve the overall situation, but now it's awfull. Need to live 2 months with old internal SATA 2 HDD. Terrible.
 
Last edited:
Feb 18, 2019
57
3
45
2
Reading from slow drives and writing to slow drives rivals watching paint dry...in the winter....!

Spend ~$200, get a pair of 1 TB Crucial MX500 drives, one for your source, another in a USB 3.0 adapter at least. Consider getting an Icy Dock5.25" front bay or two (one for work, one for home) that holds 2 or 4 of the 2.5" drives...you can copy quickly at work to get the heck out, take the drive with you, and copy to slower media later at your leisure while you watch Star Trek TNG reruns...

With both fast source and destination media, your times should be trimmed considerably...from 2 hours down to under 30 min, I'd guess/hope...
The main prob is 200S. No kidding.
 

hang-the-9

Titan
Moderator
The problem is that I need to do it during daytime and concerning better hardware you mentioned in a link, I tried to use something like this to obtain usb 3.0 but the only thing I got after Windows is loaded with such kind of card was the sign - This device can perform faster, so...
If you are doing this to archive something, why does it matter when you run it? It will copy over if you are there watching it or not. It also sounds like you are copying every file no matter if it was changed or not, do a copy of only new or changed files.

If you are doing this as a business, cost to get faster equipment should simply be part of the business. If you spend 2 hours a day watching this, you are wasting 2 hours of production time.

Or buy a cheap used computer dedicated to the file copy so you don't lock up your main system. A used system with USB 3 you can find for maybe $100 if you don't care much about the CPU in it. Even laptops with USB 3 are around in the $100 range.
 
Reactions: Garry423

Ketchup79

Notable
Aug 7, 2019
1,266
116
1,090
55
Is this your computer you use for work, or a computer owned by a company you work for? I am all in favor of helping speeding up this process, but not of putting money into a computer that isn't yours.
 
Copying data to any 5400 RPM external drive is going to be slow, no matter if it is connected via USB 2 or 3.

You can put in a 250-500 GB SATA SSD ($50-$65?)into a USB 3.0 enclosure, and one can at least eliminate the destination drive being your obvious bottleneck.

If copying from a spinning drive, now the source drive is the limit. SATA2 (Core2 days) speed limits are ~250 MB/sec, not that that would ever come into play with any spinning drives. Again, a clone to a 500 GB SSD for another $65 would go a LONG way to now making the rest of the system the slow point, where as now the storage (source and destination) almost certainly is.
 
Last edited:
Feb 18, 2019
57
3
45
2
Is this your computer you use for work, or a computer owned by a company you work for? I am all in favor of helping speeding up this process, but not of putting money into a computer that isn't yours.
It's my own. I've got a little cleaning business in hard times(((
 
Feb 18, 2019
57
3
45
2
Copying data to any 5400 RPM external drive is going to be slow, no matter if it is connected via USB 2 or 3.

You can put in a 250-500 GB SATA SSD ($50-$65?)into a USB 3.0 enclosure, and one can at least eliminate the destination drive being your obvious bottleneck.

If copying from a spinning drive, now the source drive is the limit. SATA2 (Core2 days) speed limits are ~250 MB/sec, not that that would ever come into play with any spinning drives. Again, a clone to a 500 GB SSD for another $65 would go a LONG way to now making the rest of the system the slow point, where as now the storage (source and destination) almost certainly is.
I'm looking for an upgrade during a month or two. With an SSD aboard this RIG, the reading - wrighting time was more than acceptable, but SSD smoked itself several weeks ago and I forced to use an old SATA2 drive for archiving. Now, with the helf of this forum, I make one big archive and it reduces time to almost an hour. With an SSD it was about 40 minutes. Lunch abd Coffee. No waste of time/ The CPU and components are enough for my objectives. Games I play are old, programs also, It's (like my netbook) enough for me. I was looking for a good advice and, i suppose, the best of 'em is to make one big file instead of many tiny photoes and txts. Thanks.
 
Feb 18, 2019
57
3
45
2
Copying data to any 5400 RPM external drive is going to be slow, no matter if it is connected via USB 2 or 3.

You can put in a 250-500 GB SATA SSD ($50-$65?)into a USB 3.0 enclosure, and one can at least eliminate the destination drive being your obvious bottleneck.

If copying from a spinning drive, now the source drive is the limit. SATA2 (Core2 days) speed limits are ~250 MB/sec, not that that would ever come into play with any spinning drives. Again, a clone to a 500 GB SSD for another $65 would go a LONG way to now making the rest of the system the slow point, where as now the storage (source and destination) almost certainly is.
My SATA 2 internal Drive is 7400 according AIDA64
 
Feb 18, 2019
57
3
45
2
If you are doing this to archive something, why does it matter when you run it? It will copy over if you are there watching it or not. It also sounds like you are copying every file no matter if it was changed or not, do a copy of only new or changed files.

If you are doing this as a business, cost to get faster equipment should simply be part of the business. If you spend 2 hours a day watching this, you are wasting 2 hours of production time.

Or buy a cheap used computer dedicated to the file copy so you don't lock up your main system. A used system with USB 3 you can find for maybe $100 if you don't care much about the CPU in it. Even laptops with USB 3 are around in the $100 range.
It matters bacause I cannot use PC during the writing process, but i sometimes need to, that's why I sometimes use netbook to continue without big delay. Even client database is hard to open during archiving. That's the point. When my SSD was alive I could easily work with this copy making in a background process. That startled me when I opened this topic.

My business in a hard times. Next to months just to endure, then I'll buy an SSD. Thanks.
 
Thread starter Similar threads Forum Replies Date
L Storage 7

ASK THE COMMUNITY

TRENDING THREADS