Best way for copying very high amount of files into a NAS

[hgh_it]

hi guys,

Straight to the point,i have a very high number of files which are over 2,000,000,000 Nos, the volume of each file is between 1 KB and 30 KB,and because of the enormous count of files,copying them into a NAS (Network Access Server which contains 8 HDDs that are in 10 RAID-resulting in a 6GB Drive) is a complicated task. copying through windows ordinary copy paste usually takes years(!!!) to even calculate the task progress,and i also have tried using following softwares :
UltraCopy,
TeraCopy,

above softwares usually fail in the first initial minutes,the following software-which is a great one for Copy Tasks- does not hang but again takes a very long time:
RichCopy

also i tried using Norton Ghost, which takes a image of the source drive in a short time,but again copying from the image to the destination drive is very very slow.

I'm looking for a solution which can copy the files from source HDD without worrying about dealing with every and each one of files,but only copying segments of HDD no matter how many files are located in them,perhaps this solution will result in a very faster copy.

I would appreciate if you offer me any professional solution for this issue.

Thank you!

 

[Nothing_But_NAS]

FYI.. NAS=Network Attached Storage. Servers were inherently designed to be on a network and "serve" so your acronym would be redundant.

What NAS do you have? Or what OS is your NAS running? You should look into rsync. Rsync will sync files between multiple location, copying only the files which are different. If the process fails, the next run of rsync will resume from where it left off as it only does delta copy.

 

[zdbc13]

What kind of NAS do you have (brand, model, etc.)? Is there any way to connect it directly to the pc that has the data and do a fast copy that way?

If it has to go over the ethernet make sure both devices are attached to the same router or switch. You could use robocopy: eg. will use 10 threads in windows 7

robocopy C:\myFiles F:\backup /MT:10

Or use one of these programs (I've never used them so don't endorse any one)
http://www.technobuzz.net/3-ways-to-copy-files-faster-and-easier/

 

[3p0cHx]

Certainly people will see more and more of the painstakingly slow performance of budget NAS devices as they are beginning to proliferate the business and home markets.

We are copying ~2TB from a Western Digital NAS to a QNAP with RSync. Because it looks like SSH is causing high CPU usage on the WD, we are considering 1 of 2 options: create cron job with rsync to run after hours or attempt to speed up the copy by mounting 2 nfs shares and then run rsync. Perhaps the encryption of SSH traffic is causing the copy to be slow. I’ll try to remember to post back results but please pipe up if you’ve had better success copying NAS to NAS with nfs and rsync or a different method entirely.

I’m also currently running FastCopy v2.1 to copy around 640GB to a single usb-attached sata drive. RichCopy error-ed out a couple times after a few hours. After 70GB I switched to fast copy and fast forward 8 hours, 190GB of data had copied which seems like a big improvement in performance.

What’s the ideal way to get the best performance out of these types of devices regardless of volume size and RAID type?