-
Have a passion for hardware and the written word? Tom's Hardware is seeking contributors!
HDD format vs quick format
- Thread starter vixl2ds
- Start date
Exactly. A Full Format must write to each sector, then read it back and verify it is read correctly. This must be done for EVERY Sector on the HDD, and that takes hours!
A Quick Format writes the Root Directory and creates a special set of files that keep track of the use of Sectors. But it does NOT write or read anything else.
A Quick Format writes the Root Directory and creates a special set of files that keep track of the use of Sectors. But it does NOT write or read anything else.
thanks a lot, was very insightful!
can you also tell me how might those bad sectors appear, or the very deffinition of "bad sectors".
what will happen if the drive finds them while formatting? Are they repaired or the capacity is lost?
Within the framework of Windows and its utilities like Format and CHKDSK, a "Bad Sector" is one that yields an error when it is read. When a HDD is asked for the contents of a Sector it reads it and checks some data error info. If an error is indicated, it tries to re-read the Sector and get good data. If it cannot, it reports this to the OS that asked for the data as a Read Error. In the normal operation of a Windows app, you the user would get a Read Error message and the file in question would be considered corrupted, with consequences for how useful it might be and whether you can recover most of the data and try to fix the error (often you can't).
As a separate administrative process, Windows keeps a file record on the HDD of "Bad Sectors". A Red Error of this type will not cause Windows to note it there. However, the CHKDSK and Format utilities are specifically designed to find and "fix" this kind or error. If they receive a Read Error note from the HDD, they log that Sector into the Bad Sectors file, then assign a new unused Sector to its place in the file. Windows will never try to use that Bad Sector again. BUT the replacement sector it assigned to that place in the file contains no useful data, and hence that file itself is now readable, but corrupted, so it's up to you to try to replace the file with a good copy from somewhere, or edit its contents to repair the erroneous data, or maybe just give up on the file and delete it. The handiest place for having Windows find these Bad Sectors is in a Full Format because that process destroys all old files anyway. When it is done, all the Bad Sectors are placed in the special Windows list and never used, so no new file can be corrupted by trying to use them. Of course, this does not completely prevent the possibility of a "good" Sector's developing a flaw later and becoming "bad" while it contains data.
Full Format will delete the old file of Bad Sectors and write a complete new one. CHKDSK, on the other hand, will keep the old list and add to it if necessary.
Now, a brand new HDD normally has been thoroughly checked in manufacturing, so some people do not bother to do a Full Format on a brand new HDD, but will do it later when re-Formatting an older HDD.
Modern hard drives have an additional completely separate error-correcting process that Windows and other OS's are completely unaware of. Recall that today's HDD's have microprocessors-based controller boards on board with their own BIOS, EEPROM memory and cache RAM so they can do a lot of work themselves. They have an error-correction system that basically works like this. Each time a Sector Read comes in from the OS, the quality of the electronic signal read from that Sector is assess by the HDD itself, as well as checking for data validity. If it's fine, the data is sent out as requested. If it is a weak signal but with valid data, the data is sent out again. But the HDD then replaces that weak Sector with a spare good one. (Back at the last stage of manufacturing when the true Low Level Format is done, the HDD stores the locations of all the excess Sectors it has (yes, it has more than its specified space) that are good in a table on the HDD for this use.) It then writes the known-good data from the weak Sector to the new replacement, thus restoring the file to perfect condition. The weak sector is noted in another on-disk location and never used again. If the original Read generated bad data, the HDD will try several times to re-read that data and apply error-correcting tools to try to get valid data from the poor signals. If it succeeds, the good data are sent out, and the process of replacing the Sector with a spare good one is carried out, including writing the re-constructed good data to the replacement Sector. If, however, even the re-read and analysis process fails to get good data, the HDD will report to the OS that it got a Read Error and cannot provide valid data from that Sector. It will then go through the Sector replacement process, but this time it cannot write valid data to the replacement Sector. In the meantime the OS (e.g., Windows) will do its own thing for handling a Read Error. So the modern HDD actually has a better process for Read Errors from a Sector than Windows does, and it's all invisible to Windows The HDD only reports a Read Error if it cannot fix that itself.
Now, that internal process on the HDD only happens when a Sector is Read. But there's a useful (though time-consuming) process that will force the HDD to examine ALL of itself. It is a Zero Fill - a process that writes just zeroes to EVERY Sector of the HDD currently available (but not including the hidden collection of spare Sectors). This works because by default, every time the HDD writes data to a Sector, it reads it back again and checks that it matches what was written. (This is also a part of the internal self-checking routines.) That Read process involves all of the signal analysis outlined above, as well as the data comparison. Hence every Sector filled with Zeros (and that's all of them) will get tested, and all questionable ones will be replaced by the HDD itself. When the process is finished, to the outside world (read: Windows or other OS), this HDD is perfect - it has NO Bad Sectors available.
Now, this latter internal process depends on having a stock of known-good spare Sectors to use as replacements. Over the life of a HDD, more and more will get used up. At some point the remaining stock gets so low that it MIGHT in the near future be unable to replace a bad Sector with a good one. This triggers one type of alarm message from the on-disk reporting system called S.M.A.R.T. So if your BIOS is set to Enable SMART monitoring of the HDD's, when this happens you will get a warning that the SMART status of the HDD has failed, and it will be up to you, the user, to find out exactly what type of SMART error warning has come up. That usually means using a diagnostic utility. If you get a SMART message saying that the cumulative faulty Sector count of the HDD has exceeded the set limit, that's a warning that the HDD is still working fine now, but MAY not be able to keep fixing itself for much longer. That is when you buy a new HDD and clone the old one to it, BEFORE the HDD actually fails and has data corruption to deal with.
Long story, maybe with more info than you wanted. But it's interesting just how much behind-the-scenes stuff is going on the prevent data loss.
As a separate administrative process, Windows keeps a file record on the HDD of "Bad Sectors". A Red Error of this type will not cause Windows to note it there. However, the CHKDSK and Format utilities are specifically designed to find and "fix" this kind or error. If they receive a Read Error note from the HDD, they log that Sector into the Bad Sectors file, then assign a new unused Sector to its place in the file. Windows will never try to use that Bad Sector again. BUT the replacement sector it assigned to that place in the file contains no useful data, and hence that file itself is now readable, but corrupted, so it's up to you to try to replace the file with a good copy from somewhere, or edit its contents to repair the erroneous data, or maybe just give up on the file and delete it. The handiest place for having Windows find these Bad Sectors is in a Full Format because that process destroys all old files anyway. When it is done, all the Bad Sectors are placed in the special Windows list and never used, so no new file can be corrupted by trying to use them. Of course, this does not completely prevent the possibility of a "good" Sector's developing a flaw later and becoming "bad" while it contains data.
Full Format will delete the old file of Bad Sectors and write a complete new one. CHKDSK, on the other hand, will keep the old list and add to it if necessary.
Now, a brand new HDD normally has been thoroughly checked in manufacturing, so some people do not bother to do a Full Format on a brand new HDD, but will do it later when re-Formatting an older HDD.
Modern hard drives have an additional completely separate error-correcting process that Windows and other OS's are completely unaware of. Recall that today's HDD's have microprocessors-based controller boards on board with their own BIOS, EEPROM memory and cache RAM so they can do a lot of work themselves. They have an error-correction system that basically works like this. Each time a Sector Read comes in from the OS, the quality of the electronic signal read from that Sector is assess by the HDD itself, as well as checking for data validity. If it's fine, the data is sent out as requested. If it is a weak signal but with valid data, the data is sent out again. But the HDD then replaces that weak Sector with a spare good one. (Back at the last stage of manufacturing when the true Low Level Format is done, the HDD stores the locations of all the excess Sectors it has (yes, it has more than its specified space) that are good in a table on the HDD for this use.) It then writes the known-good data from the weak Sector to the new replacement, thus restoring the file to perfect condition. The weak sector is noted in another on-disk location and never used again. If the original Read generated bad data, the HDD will try several times to re-read that data and apply error-correcting tools to try to get valid data from the poor signals. If it succeeds, the good data are sent out, and the process of replacing the Sector with a spare good one is carried out, including writing the re-constructed good data to the replacement Sector. If, however, even the re-read and analysis process fails to get good data, the HDD will report to the OS that it got a Read Error and cannot provide valid data from that Sector. It will then go through the Sector replacement process, but this time it cannot write valid data to the replacement Sector. In the meantime the OS (e.g., Windows) will do its own thing for handling a Read Error. So the modern HDD actually has a better process for Read Errors from a Sector than Windows does, and it's all invisible to Windows The HDD only reports a Read Error if it cannot fix that itself.
Now, that internal process on the HDD only happens when a Sector is Read. But there's a useful (though time-consuming) process that will force the HDD to examine ALL of itself. It is a Zero Fill - a process that writes just zeroes to EVERY Sector of the HDD currently available (but not including the hidden collection of spare Sectors). This works because by default, every time the HDD writes data to a Sector, it reads it back again and checks that it matches what was written. (This is also a part of the internal self-checking routines.) That Read process involves all of the signal analysis outlined above, as well as the data comparison. Hence every Sector filled with Zeros (and that's all of them) will get tested, and all questionable ones will be replaced by the HDD itself. When the process is finished, to the outside world (read: Windows or other OS), this HDD is perfect - it has NO Bad Sectors available.
Now, this latter internal process depends on having a stock of known-good spare Sectors to use as replacements. Over the life of a HDD, more and more will get used up. At some point the remaining stock gets so low that it MIGHT in the near future be unable to replace a bad Sector with a good one. This triggers one type of alarm message from the on-disk reporting system called S.M.A.R.T. So if your BIOS is set to Enable SMART monitoring of the HDD's, when this happens you will get a warning that the SMART status of the HDD has failed, and it will be up to you, the user, to find out exactly what type of SMART error warning has come up. That usually means using a diagnostic utility. If you get a SMART message saying that the cumulative faulty Sector count of the HDD has exceeded the set limit, that's a warning that the HDD is still working fine now, but MAY not be able to keep fixing itself for much longer. That is when you buy a new HDD and clone the old one to it, BEFORE the HDD actually fails and has data corruption to deal with.
Long story, maybe with more info than you wanted. But it's interesting just how much behind-the-scenes stuff is going on the prevent data loss.
yeah, thanks a lot for this. I enjoyed reading it. It's really fascinating how things work.,, just yesterday I found out that a format isnt a way to completely erasing data, and that it can be recovered with special tools (that you need a disk wipe to make it unrecoverable). Will read more of that when I have the time.
You are right about Formats - sort of.
In ANY version of Windows, a Quick Format does NOT write to most of the HDD - it only writes a new Root directory and Sector-tracking files - so any old data is still there, it's just hard to sort out. But good software can do it.
Current versions of Windows, however, DO obscure old data very well when you do a Full Format because they WRITE a particular pattern to a Sector and then read it back and check for errors. That makes recovery of any FORMER data that has now been overwritten extremely difficult. It's not impossible - some very sophisticated software and a HUGE amount of skilled operator time can get most back, but very few have the time, tools and skill to do that - certainly NOT an amateur, even a sophisticated computer user. Interestingly, though, older versions of Windows (XP original version without Service Packs, maybe?? and earlier) did NOT write their own data to start a Full Format. They READ the existing data and saved it, then wrote and read their own test data, and finally restored the original data to each Sector. By using those extra steps (costs time!) they left old data that COULD be recovered in the same way that they can be from a Quick Format.
As I said, the current Full Format method makes it ALMOST impossible to recover old data. The same can be said for using a Zero Fill process because it over-writes the old data with a new pattern everywhere. Although experts with huge time and tool resources (think spy agencies) might get old data back from an HDD treated by either of these, in practice they both destroy old data beyond any reasonable recovery effort. To be POSITIVE of data destruction one can buy and use software to do a HDD wipe using what's known as military-grade wiping. This involves writing to every sector a series of specially-designed patterns that repeatedly reverse the magnetic signal in the surfaces so that, when it is done, it really is impossible to recover traces of the original data. OR, you can go low-tech: physically smash the HDD unit to pieces with a large hammer!
In ANY version of Windows, a Quick Format does NOT write to most of the HDD - it only writes a new Root directory and Sector-tracking files - so any old data is still there, it's just hard to sort out. But good software can do it.
Current versions of Windows, however, DO obscure old data very well when you do a Full Format because they WRITE a particular pattern to a Sector and then read it back and check for errors. That makes recovery of any FORMER data that has now been overwritten extremely difficult. It's not impossible - some very sophisticated software and a HUGE amount of skilled operator time can get most back, but very few have the time, tools and skill to do that - certainly NOT an amateur, even a sophisticated computer user. Interestingly, though, older versions of Windows (XP original version without Service Packs, maybe?? and earlier) did NOT write their own data to start a Full Format. They READ the existing data and saved it, then wrote and read their own test data, and finally restored the original data to each Sector. By using those extra steps (costs time!) they left old data that COULD be recovered in the same way that they can be from a Quick Format.
As I said, the current Full Format method makes it ALMOST impossible to recover old data. The same can be said for using a Zero Fill process because it over-writes the old data with a new pattern everywhere. Although experts with huge time and tool resources (think spy agencies) might get old data back from an HDD treated by either of these, in practice they both destroy old data beyond any reasonable recovery effort. To be POSITIVE of data destruction one can buy and use software to do a HDD wipe using what's known as military-grade wiping. This involves writing to every sector a series of specially-designed patterns that repeatedly reverse the magnetic signal in the surfaces so that, when it is done, it really is impossible to recover traces of the original data. OR, you can go low-tech: physically smash the HDD unit to pieces with a large hammer!
TRENDING THREADS
-
-
My week with Linux: I'm dumping Windows for Ubuntu to see how it goes- Started by Admin
- Replies: 232
-
Discussion What's your favourite video game you've been playing?- Started by amdfangirl
- Replies: 4K
-
Discussion PSU recommendations and power supply discussion thread - Tom's hardware- Started by Darkbreeze
- Replies: 2K
-
-
News Nvidia RTX 5060 is up to 25% faster than RTX 4060 with frame generation in new GPU preview
- Started by Admin
- Replies: 47
Facebook
Twitter
Instagram