Copying Files from HD with Bad Sectors

mspy

Distinguished
Feb 26, 2013
5
0
18,510
I have 2 WD Green 1.5TB SATA2 that I bought around 3 years ago for storage, apart from both having a high amount of 'load cycle count' around 150.000 due to WD parking the heads every 8s of inactivity I think, which according to WD itself it isn't anything to worry about, now they both started reporting bad sectors in days of each other, or maybe the bad sectors were always there I do not know since only now I almost filled up both drives with data. Next time I buy a new drive what tests should I run in order to see if they came in good condition or not?

Anyway moving on, I will buy a new HD in a week and I hope to copy all the data from this bad drives to the new one, my question is what is the best way to do this, I mean both the drives seem to work atm, but I wonder if doing a simple copy-paste with windows 7 will it copy files that are corrupted due to bad sectors or if the copy goes through with no errors prompted I can safely assume that all files were copied properly with no errors at all? If not how can I tell which files were corrupted afterwards I copied everything? If this is not possible to know is there any software that can at least tell me what files are located on a bad sector and have a good chance of being corrupted so I can try to fix the files on my own?

One last thing, what software is recommended to try and fix bad sectors? and should I try to fix the bad sectors before or after I backup all my files? I heard that if I try to fix bad sectors before copying theres the possibility that this sectors will be marked with 0 and all data is lost, but other people say that before it marks with 0 all the data are transfered to a good sector first so it will not be lost, which is correct?
 
Hi,

From my experience their are a few programs that might help to copy but none will work 100%.

my best luck is with unstoppable copier

http://www.majorgeeks.com/Unstoppable_Copier_d2623.html

But if you see it's really slow and takes a lot of time it's probably useless (many tries on same sector, time not moving)

The best way I found to repair a drive with bad sectors is low level format with ex. seagate low level format.

About 33% of drivers come back with 100 % health 100 % performance while the rest could not be saved.

Some of the repaired one sometime act funny in windows but still worked properly.

Good luck
 
I would disconnect them for now, and when it comes time to copy the data off, what I do usually is small copies folder by folder. Then if you run into a bad sector or CRC error when it stops copying, it's easier to know where. If you just say right click and say copy movies and that has 1000 1GB movies, it could fail and you wouldn't know where. Select 10 movies at a time and copy instead, then give the PC a few min break, then copy some more. I've had anyways much better copying data off with small copies than trying to select 1TB at once and copy.
 
When you say they have "started to report bad sectors", that's not clear what report you are seeing. In some cases, you are lucky and the HDD itself has made sure the data are safe and the bad sectors are not in use. However, that is not guaranteed.

If there really is a bad sector in use holding data from a file, there is no way to get good or repaired data from that sector. Even if the copying system does copy the file, it will be corrupted. Generally, in fact, you are better NOT to succeed in copying a corrupted file with a bad sector. You are better knowing exactly what file that was, and finding a different way to get a good copy of it. (In certain cases, you actually might be able to use a corrupted copy of a file and fix the error yourself - an example might be a text file where you can dead it and edit out the error. But most files won't co-operate this way.)

Now, how do you know? That's the tough part, and the reason for getochkn's recommendation. The normal "Copy" command or drag-and-drop process will proceed merrily along copying a whole bunch of files until it hits an error. Then it quits and tells you it quit for that reason, BUT it never tells you which file had the error, and which files already were copied successfully before the problem! So if you do only a few files at a time and this happens, you can backtrack and try to copy one of those files at a time to see which is trouble. Then you go back to the few-files-at-once process until the next error. If you are lucky this will take a long time BUT you will know exactly what files need to be replaced.

Now, you CAN try betting that so far, NONE of your files are corrupted. You copy LARGE numbers or groups of files at a time, and if nothing generates an error, you've just saved a LOT of time. It's when an error does happen that you have to start working on small groups to identify the problem and copy the rest.
 


Doesn't this program and also the method of 'reparing' the disk with the manufacturer software (in this case Western Digital Lifeguard I believe it's called) replace the corrupt parts of a file with 0 making it not a exact copy of the original file? How do I know if this happens, does it reports on a log or something?



Well the first disk I notice it had bad sectors after hanging up when I opened a folder with around 200 pics, then I checked HD Tune and it was reporting this:

- Current Pending Sector: 289
- Offline Uncorrectable: 274
- Write Error Rate: 284

At the time I checked the other disk too and it was ok but 2 days later it was not ok anymore, when I booted the computer I got something like this on post screen 'Smart Event Logged' that was when I checked it again and saw this for the 2nd disk:

- Current Pending Sector: 53
- Offline Uncorrectable: 48
- Write Error Rate: 1969

I don't know what caused this, both disks are the same model and I connected them to the PC around the same time 3 years ago. I use both individual for storage mainly. I read somewhere that malware or RAM faults can cause bad sectors on disk and I tested my RAM with memtest and it seems OK, as for malware I use Bitdefender 2010 for ages now and I always run it for updates.

Even if I use a tool like HDD Regenerator won't I be able to retrieve all the data from a bad sector? I think this tool does not replace bad sectors with '0' and tries to revive it instead hence all data could be copied but I might be wrong on this.

Well if I know which files are located on a bad sector that alone would indeed help, but seriously copy-pasting all files one by one of a 1.5TB drive is troublesome, I wish there were an easier method, a program of some sorts, that could just detect the corrupted files and report them.
Just one thing, if a normal windows copy-paste goes fine with no errors, is it grantee that the files have a perfect copy on the destination drive, or is there a possibility that a file was copied already corrupted? I mean in a case like the HD firmware remaps bad sectors wouldn't the remapped data from the bad sector be forever lost and windows will not know that this is the case and would think the data is 100% good or remapping a bad sector transfers all the data from it to a good one on the first place? This is why I asked about if I should try and copy all my files first before trying to repair the disk with some kinda of a tool, because most of them seem to repair the disk by filling bad sectors with 0 and I do not know if this is a good thing or not, I do not know if this makes the data of a bad sector inaccessible forever thus not possible to recover anymore in their perfect condition.
 
If you try to copy a file that has corrupted data in one Sector, it is nearly impossible for Windows to NOT see the corruption and throw an error message. (Since a Sector is typically 512 bytes, almost all files consist of MANY sectors.) Within each Sector there are a few mathematical items calculated from the actual data. Every time the Sector is read, those items are recalculated from the data and compared to the values stored in that sector. If even one bit of data has changed, those checksums etc. will not match, and that will be recognized as an error. So your chances of unknowingly copying bad data from a Sector to another place are virtually none.

There is one way that a corrupted file might be copied. When you use some diagnostic tools like CHKDSK and they find a Sector with corrupted data, their action is part helpful, part problem. When a Bad Sector is found, the system updates its on-disk table of such Bad Sectors to include that one so it is never used again. IF you have told CHKDSK to correct any errors it finds, it assigns a new known-good Sector to fill in that gap in the file. Unfortunately, Windows does not have any good tools for trying to extract good data from a Bad Sector, so the substituted Sector does NOT contain any data from the original Bad Sector. Technically it is valid data and can be read without errors, so the "fixed" file can be read and copied. HOWEVER, that "fixed" file contains a Sector of random data and is thus corrupt.

Completely hidden from you (and from Windows), a modern hard drive itself does a better job in the background. Every time it reads a Sector it makes an assessment of the signal strength it gets. If the signal is getting weak but the data itself is good, the HDD will copy that data to another known-good spare Sector, record the weak Sector in its own "Do Not Use" table, and put the new good sector into the file records. If the data tests out as faulty, the HDD will go back and re-read that Sector many times, then apply a selection of software techniques to try to make sense of the results so the data are corrected. If that works, it can write that recovered data to a good spare Sector and proceed with the substitution as above. The third possibility is that even that data recovery process fails and the HDD concludes that it cannot get reliable data from that Sector. THEN it tells the OS (Windows) that a Read Error has occurred, and lets Windows deal with that.

In doing all this background checking, the HDD itself is much better-equipped than Windows to handle questionable Sectors. Further, as it does this, every time it has to make a substitution of a spare good Sector for a questionable one, it keeps track of how many replacements it has made. It has a limited number of spare good ones to use. So at some point the S.M.A.R.T. monitoring system in the HDD sends out a warning message that its stock of good spares is getting low because it has done a lot of replacements. When you get that message, it is entirely possible that ALL of the replacements so far have been completely successful in recovering data from weak Sectors, and NONE of your files are actually corrupted yet. BUT it is a warning that the HDD is getting worse, and it is definitely time to replace the drive and copy all your data off it while you still can.

So, that's why I asked what kind of information you had that led you to understand that the HDD's have Bad Sectors. IF your info is based mainly on the SMART warning system, you may not yet have any corrupted files, and you may be able to copy everything to a couple of new HDD's. Just do that soon! On the other hand, if there have been other events including warnings from Windows of Disk Read Errors, you may have a few corrupted files. But as I said, when you go to copy everything, those corrupted files will generate errors and won't be copied. The only likely way you might copy a corrupted file is if you have used CHKDSK to find Bad Sectors and told it to "fix" those errors by plugging blank good sectors into the affected files, thus making them readable without errors, but corrupted anyway.

Bottom line certainly is that you should take action to replace your HDD's and copy all their data to the new HDD's, as soon as you can. Sometimes you're lucky and it all goes smoothly. I had a HDD fail on me about a year ago, alerted by a SMART message. I got a new HDD and simply ran a cloning software package to clone from failing unit to new unit. The cloning software alerted me to many faulty Sectors because it was trying to copy EVERY Sector (not just the ones containing files), and I had to tell it to ignore those problems and just keep on copying. The resulting clone works perfectly - as far as I can tell, no files actually were corrupted.
 
This is very good info Paperdoc! I did not run CHKDSK yet because I was afraid that it was gonna fix-by-corrupting my data with 0 as you said, but in case I did could it be unfix or this fix is permanently. As far I understood this CHKDSK 'fix' only happens at the OS side of things so if I was to run a software that could detect and ignore this 'fixes' of windows OS or even have or install another OS on the same or another drive, those 'fixed' sectors would appear to be not fixed at all and thus I would be able to try and recover 100% of the data since the true and permanent fix only happens at the drive firmware side of things which never replace the corrupted data with 0 when it is unable to recover it am I right, assuming that I do not move or edit the fixed by CHKDSK files I suppose, as if I do manually move those fixed files the corrupted part of it that was not copied by CHKDSK and replaced by 0 instead in order to do the fix is gone forever. Btw if this is the case does a Virtual Machine OS share these CHKDSK or other tool 'fixed' pool of sectors with each other or is it completely independent?

Another issue that bothers me for future reference is the warnings that a drive is falling. At the time I as not running any specific tool on the background for this purpose, I only checked HD Tune SMART info after I notice that windows was hanging when I opened one folder with lots of pics inside. Only then I had the knowledge of what was really happening. After that I seem to got a warning when I was booting up the PC and it was doing the post screen there was a line below one of the drives when it was listing them that said about some SMART event being logged, but in this case it was on yet another drive. This made me check the SMART info on HD Tune of said drive and it was indeed also failing, fumy thing is that the day before it was reporting fine results. So my question here is what kinda of warning system windows or BIOS have by default if one at all, in case of having any, what software I should keep running on the background to give me this warnings automatically?

Finally let me tell that I disconnected both my impending failing HD from the MB and PSU until I buy a new drive which should happen next week or so to backup my data. I will need to buy a 3TB HD in order to have enough space to migrate all the data there. I was checking around and notice that the WD Caviar Green drives are once again the ones cheap and at hand on most of my local stores but I am now really reluctant to buy a new Green drive from WD after all that happened to me, I do indeed have 2 other 640GB drives from WD that are going strong and are connected to the PC even more time than the 2 that failed but these drives are from the Blue line which don't seem to have any above the 2TB mark. At the time I bought the Greens mainly for storage purposes because they seemed a good deal and perfect for the job, now I'm not so sure. Should I place my luck once again on these Green line of WD or go for another brand, if I were to go to another brand which one should I go for? Some of the local stores here also have Seagate drives which seem good with 1TB platters making a grand total of 3 for a 3TB drive, whereas the WD has 4 platters I think. A read somewhere that WD Green series have a problem with load cycle counts since it parks the heads every 8 secs or so of inactivity, in my case however the SMART info I got from it was around 150 000 cycles counts thou significant higher from the Blue ones that I got which have around 2000 (the one I use for OS) and 7000 (the one I use for storage) it is not nearly has a million cycles like some people reported.

 
Chkdsk is a Windows application. It (and Windows itself) keeps the Windows list of Bad Sectors in a file on each respective drive. That is, the C: drive has a file on it with Windows' list of Bad Sectors for that drive; the D: drive has its own list on it. These files are basically "hidden" so only Windows uses them normally. However, they are no big secret, so anybody who writes their own utility to do some diagnostics and repairs on a HDD under Windows might refer to that list.

No software could detect and ignore the "fixes". Or at least, it would be VERY difficult! I don't even know the details of how the NTFS File System works - I understand nobody outside Microsoft REALLY knows that. So what I'm about to outline is NOT exactly what happens, but it will give you some feeling for the complexity. This outline is from the older FAT32 system. The NTFS File System uses more files and tracks much more data in greater complexity.

In FAT32, there are two fundamental file structures at the base of the system. The first is the Root directory, which has space for up to 254 files, and that's all. For each file, it has a number of entries in "fixed fields" - that is, the length of each entry is rigidly defined. One file entry has things like the file name, the file extension, the number of Sectors used for the file, the file length in bytes, the file date, and the Sector Number of the very first Sector it uses. If a file is truly deleted, its name is modified by changing only the first character to one particular reserved character. Now, 254 files is not much! But the system allows you to create Sub-Directories (now called Folders) of unlimited size. Each Sub-Directory is actually simply a special kind of file structured in the same way. But since a file can be of any length, so can a Sub-Directory and it can have a VERY large number of files (and Sub-Sub-Directories, etc.) in it. So that's how the START of each file can be found, along with estimates of its length.

The other key structure is known as the FAT or File Allocation Table. It has a fixed size, and every entry in it is just the number of a Sector. Hence this system imposes a limit on the size of the HDD, because it limits the number of Sectors it can track, and a Sector is 512 bytes. That is one of the reasons that it is rarely used these days. Within the FAT, the entry that corresponds to one Sector on the HDD holds the Sector Number of the NEXT Sector that is part of this file. Remember that the Directory entry points to the file's FIRST Sector; from then on each Sector points to the next. The LAST FAT Sector for a File has a special number it it that signals the End of File. When a file is truly deleted, all of its entries in the FAT are reset to 0, so a string of zeros in successive FAT table positions indicates that a file USED to occupy those Sectors, but now it is available for re-assignment.

As a HDD is used and files are created, the Directory and Sub-Directories are filled up with entries, and the FAT is also filled up to track Sector use. When deletions happen, these are revised, leaving unused FAT positions indicating Sectors are available. As more files are created, those Sectors MAY be assigned to new files and filled with new data, and the corresponding entries in the FAT are updated. At first, when new files are created, it is likely that the system will choose a block of never-used Sectors to give that file, and the FAT entry for the file will look like each entry just points to the next Sector on the HDD. But after a while those blocks of empty Sectors are all broken up, and the FAT entries start to look like the file is scattered all over the HDD. The sequence in the FAT has no pattern at all. This is called Fragmentation.

So now imagine what happens when CHKDSK decides one Sector is Bad, AND you have told it to "fix" that problem. It writes that Sector Number to its secret file of Bad Sectors on the HDD and never uses it again. Then it goes back and finds a Sector not in use, and assigns that to the gap in the file, and writes that Sector Number to the correct FAT position. Much later some hypothetical software tool comes looking for files that have been "fixed" in this way. By scanning through the Directory and FAT it can identify the START of each file's trail in the FAT, and it can check whether each FAT entry points simply to the next Sector in sequence. If it finds one NOT in a simple sequence, it could assume that was a substitute. BUT -and it's a HUGE BUT - that out-of-sequence entry could easily have been from a normal operation and NOT a "fix", and it has no way of knowing which is the real cause. Even if it assumes this was a "fix", it STILL does not know where the original Sector was that had faulty data. Now, MAYBE since it knows what Sector number should have been in that sequence (IF the sequence had been maintained undisturbed) it could go looking for an entry like that in Window's Secret file of Bad Sectors and try to find that Sector. THEN, even if it gets that far, it has to try to recover data from a Sector that is known to have yielded bad data before. (Moreover, in modern HDD's, the HDD itself already tried valiantly to recover data from it before letting Windows know it could not.) So what are the chances that this hypothetical utility can recover data from that? Just about NONE!!

Bottom line is that when you have lost data due to a Bad Sector, even if the file has been "fixed" so it is readable with one corrupted Sector, your chances of recovering the original data are: impossible. You are MUCH better off trying to find another copy of that file from somewhere else, or just resigning yourself to loss of the file.

Now to better news - the SMART warnings. As I said, one type of warning you could see is one from Windows reporting a Read Error on a drive, and these usually indicate a real possibility of damaged data. But much before that happens, the HDD itself is running its own data quality monitor and trying to recover data before it is really lost permanently. It is also running its own routine diagnostic checks on itself and logging the results into a set of files it keeps written on itself. These files are accessible through the SMART reporting system on the HDD.

When do you get to see SMART warnings? Usually you have two opportunities, BOTH run by the BIOS and not by Windows or another OS. First, every time your system boots, one of the things the BIOS does is to ask each HDD for its SMART report, and then it prints a brief message on the screen as part of the start-up displays. Most times you won't notice it because it says something comforting like "SMART data OK". You can turn this checking and message off in BIOS Setup, but most people leave it on, and that's good.

Of course, any application software, like HD Tune, can also ask each HDD for a SMART report and show you all it gets back in detail - often a hundred lines or so. And those are the two types of messages you report seeing. Usually the best use of the SMART reports is to warn you that small failures have been happening more often, indicating an increased probability of future failure. So your prudent response is to replace the HDD while the failures are still minor enough to allow automatic data recovery (already done by the HDD) so NO files are lost.

Regarding choosing a new HDD, I can offer a few comments. First, check what OS (version of Windows) you are using. A 3 TB HDD is larger than you can use with the Partitioning system commonly used, MBR. But that is limited to a Partition no larger than about 2.1 TB. To use it as one large drive, you need to be using a different system known as GPT, which XP does not support. Now, I think there are ways to use a 3TB unit in Win XP if you use it to create TWO Partitions each less than 2.1 TB, but I'm not sure of the details. On the other hand, if you are using VISTA or something more recent, that is not an issue.

HOWEVER, there is another issue. Among the versions of Windows, the common 32-bit versions do NOT allow you to BOOT from a HDD Partitioned in the GPT system. All 64-bit current versions of Windows do. So, if your plan is to use a 3TB unit for data storage only you'd be all right, but if you want one of these as your BOOT drive, you MUST be running a 64-bit version of Windows, and preferably Vista or later.

Regarding the WD lineup of HDD's, my impression is that the Blue line are the low-cost base of the product line. The Green ones are maybe better quality and have better features, and are specifically designed to minimize power consumption (and hence heat generation) with a number of tricks like slower rotational speeds and slower access, and maybe powering down after a period of non-use. These are very popular and well-priced, and I do not recall seeing anything to indicate they fail any more than others. Above those are the Black line that offer faster speeds and data access rates, plus other features to enhance performance, and sometimes longer warranty periods. They are hence more expensive. Beyond that are the Red line designed for use in servers and RAID array systems, and not really what you need.

There are other HDD makers, of course, and Seagate certainly is a major competitor in this area. That company had a problem several years ago with one particular new model, but I have not heard of any significant problem with Seagate units recently. Like WD, Seagate has a "green" line as well as a higher-performance line, plus some specialty designs. Actually, my machine has Seagate drives, but we do have WD units in other machines here. I'd consider both company's products reliable.

Since you have seen SMART warnings through HD Tune, let me suggest something. Run HD Tune again on your drives, and print out the results. Now, shut down the system and disconnect power. For each drive, go to the data cable plugged into the back of the drive and carefully unplug it, then plug back in. Do this two or three times. Now do the same thing for the other end of the cable that plugs into the mobo. Now go back to the drive and do the same thing for the wider power supply connector at the drive. Lastly, look around carefully and verify that all the cables you handled are reconnected properly, AND that you did not disturb some other connection in the process. Why do all this? Well, one of the reasons you can get bad data reads is the the connection between a cable and a socket is loose or is dirty with a bit of oxidation of the metal contact. Unplugging and reconnecting can "scrub" the metal surfaces cleaner and solve that problem. If this actually is NOT the source of your troubles, this process will do no good at all, but it is easy, harmless and free to try.

Now close up and reconnect power, boot up, and re-run the HD Tune tests and see if anything changed. When you're done, you might want to disconnect that one drive you already have been NOT using to preserve it.
 


Dear, Paperdoc. First of all, thank you very much for dealing with this issue thoroughly. I have been looking for such a proper clarification since the same idea came to my mind a while ago.

In terms of your second paragraph begining with the words "There is one way that a corrupted file might be copied. When you use some diagnostic tools like CHKDSK and they (...)", I wonder whether ERROR SCAN in HD TUNE (or Verify Test and Read Test in HDDscan) can affect the hard drive in the same way as you described.

Do I suppose correctly the following situation ? If the hard disk surface test completes successfully, then bad sectors are no longer used by the hard disk and instead, a spare area is used for all reads and writes targeting those bad sectors, which means consequently that a corrupted file might be copied without revealing its corruption.
 
I have not use HD TUNE or HDDscan, so I cannot tell you how they do their work.

Your third paragraph is correct, mostly. The small difference is this. If there is a Bad Sector among those of a current file, any attempt to read it (including a copy operation) will generate an error in Windows and the operation will be cancelled. However, it someone has run the CHKDSK utility BEFORE this read is attempted, any Bad Sectors will have been detected already and replaced with known-good sectors. So now all the Sectors of the file will be readable cleanly and will not generate any errors during the read or copy operation. BUT those replacement sectors that read cleanly actually contain random junk data, and NOT any of the data from the Bad Sector they replaced. So the file can copy with no problem, but it is contaminated in one spot by junk data. My small distinction from your description is just about timing sequence - the replacement Sector has PREVIOUSLY been assigned to that file by a CHKDSK operation; it does not happen at the time of the copy attempt.