[SOLVED] Faster File Transfer between two Network HDDs (WD MyCloud)?

Jan 21, 2020
2
0
10
I have a 3TB and a 4TB MyCloud attached to the same switch within my home network (Ethernet, not USB, attachments).
Is there a way to increase transfer speeds when I'm moving files between them?
I tried setting up FTP using FileZilla, but don't have enough knowledge to set up the program and definitions properly (I couldn't see the network drives - they're not computers).
Is this just stupid? Should I be using only one network drive? Set up a RAID for redundancy? I've already had one 3TB drive fail, and it cost a LOT of money to retrieve the irreplaceable files (three software solutions only made the retrieval harder).
Sorry, lots of questions in one question!
 
Solution
A "backup" is a second (or third) copy of your data. Not the only copy.
A dead drive should never result in lost data.

There is no magic to speeding up data transfer over the LAN.
Nor is RAID a viable solution for backups.

Diddly

Distinguished
You are limited by the speed of the hdd drives and for transferring data over the network you need at least a gigabit network to get decent speeds. Some NAS systems have dual ethernet ports to make transfers faster. You should consider a NAS with RAID 5 for redundancy (at least 4 drives where it can tolerate one drive dying without losing data). 4x4TB will give you 12TB of RAID 5 storage. Or use the cloud?
 

USAFRet

Titan
Moderator
A "backup" is a second (or third) copy of your data. Not the only copy.
A dead drive should never result in lost data.

There is no magic to speeding up data transfer over the LAN.
Nor is RAID a viable solution for backups.
 
Solution
Jan 21, 2020
2
0
10
There is no magic to speeding up data transfer over the LAN.
Thanks for your response. So moving data within a HDD gives speeds of 120 MB/s while between network drives can slow to 100 kb/s, and this is normal?
(Yes, I have a gigabyte network with CAT5e or CAT6 cables).

Nor is RAID a viable solution for backups.
Wow. First I've ever heard that redundancy isn't viable. Your solution is terabytes of cloud storage, then?
 
Part of your issue is the data must be transported over the network twice. It does not copy directly from driver to drive it copies from the first drive to your pc and the from your pc to the second drive.

It has to be the mycloud device but why is hard to say. Most vendors that sell NAS devices have charts that show the transfer speeds of all the different models and difference based on things like file size. These devices just talk about how you can play your music or stream your video. They don't even mention performance and I actually tried to find the data on their site. Fairly typical marketing to consumers who don't like math they just want pretty pictures.

You are thinking too simplistically if you think redundancy in raid is a backup solution. It only protects against a hardware failure it can do nothing to solve a software or user error that deletes the files.
 
  • Like
Reactions: MuscleWhisperer
Thanks for your response. So moving data within a HDD gives speeds of 120 MB/s while between network drives can slow to 100 kb/s, and this is normal?
(Yes, I have a gigabyte network with CAT5e or CAT6 cables).


Wow. First I've ever heard that redundancy isn't viable. Your solution is terabytes of cloud storage, then?

So a drive breaks you can rebuild the array, the data gets corrupted how do you restore it. NO! Raid is not a backup solution. a complete copy on separate installation and another off site is a backup solution. Yes cloud storage is a solution, AWS Glacier or the like, you only really get charged when you restore the data, at which point you wouldn't mind paying for it. A copy at your friends house is better practice than thinking RAID is a backup!!
 
  • Like
Reactions: MuscleWhisperer

USAFRet

Titan
Moderator
Thanks for your response. So moving data within a HDD gives speeds of 120 MB/s while between network drives can slow to 100 kb/s, and this is normal?
(Yes, I have a gigabyte network with CAT5e or CAT6 cables).


Wow. First I've ever heard that redundancy isn't viable. Your solution is terabytes of cloud storage, then?
RAID 1 is drive redundancy.
Not data redundancy.

If you need actual 24/7 uninterrupted ops, then a RAID 1 may be the ticket. It also needs to be supported by a comprehensive backup routine.

Which does not have to be "cloud".
A local NAS box or external drive works just fine.

 
Thanks for your response. So moving data within a HDD gives speeds of 120 MB/s while between network drives can slow to 100 kb/s, and this is normal?
(Yes, I have a gigabyte network with CAT5e or CAT6 cables).

You have a "gigabit" network, not "gigabyte". 1byte = 8bits, so a gigabyte network would be an 8gigabit network.

We also have to keep in mind that the processors of your WD mycloud devices are cheap ARM processors. They probably can't handle large file directory transfers very well.

The fastest way to do data backups between drives is probably to use Rsync so they copy directly to each other. At least that's what I found with QNAP nas units i've backed up. They averaged over 100MB/s (FTP) for the entire >5 terabyte backup transfer the few times i've done it. I used FTP though as it was twice as fast as SMB(samba) with their cheap ARM nas's when doing whole NAS transfers. I averaged about 60-70MB/s using samba.