[SOLVED] Initializing a HDD in Windows - low level description

[rchamlen]

This has bugged me for years - but never really chased it down .... but tonight I have yet another new HDD where I am getting the "need to initialize disk" message in Windows (W10). I DO know how to do this .... but my question is different (and I am amazed I do not find a clear answer in trying to search.....).

So .... at a low level, just what terminology is Windows warping to call this step "initializing" (with "formatting" following......

Historically, I come from a Unix perspective, where "formatting" a disk is an extremely low level activity performed on a totally blank disk .... initially literally demarking the locations of sectors and blocks on the disk ..... later becoming demarking equally sized blocks in a spiral from the inside to the outside of the disk. At this point no partitions or partition table are present. the next step then would be to write the partition table (exactly what this is then depending on whether the disk is a "bootable" disk containing a boot sector, or whether the disk is to be data only, containing only space reserved for the the partition table, and the partition table itself .... but the rest of the disk being then totally blank, except for the demarkation data which identifies the locations of the sectors. after this the partition table can be modified to identify partitions on the disk, and only after this can a filesystem be built within a given partition (the building of the filesystem really being not much more than several sectors of pointers which are to be used to identify the physical location (on the disk) of individual files.

In this sequence, there is no one step that has historically been called "initializing" the disk.

So .... what Windows (in their wisdom) calls "formatting" a filesystem is really just overwriting the filesystem pointer sectors - I get that.

So - in this lower level description, just what exactly is Window's "initialzing" doing?????? My guess from the sequence, and data that are available within both the "disk management" tool and the "diskpart" utility is that all windows is really doing at this step is to write (or overwrite) the partition table , and maybe creating a boot sector at that time (if you choose a bootable disk).

for those that have had a chance to really dig in to this ..... is my my guess correct?? (not critical certainly, but this has been bugging me for a while.....

 

[lvt]

I think it simply mean you have to start from scratch, for example destroy all the existing partitions and create them again then format...

It's more convenient to do that from a working computer using a tool like Minitool Partition Wizard, or any other similar tool.

 

[Alabalcho]

I'd just say that your Unix analogy is about 25-30 years outdated (unless you think about floppy / ZIP drives, where it might be still relevant). OS does not define physical sector boundaries, writes markers etc - all this is done by the disk controller. All the OS sees is CHS, and it is up to the disk controller to map CHS addresses into physical head movement / selection / waiting for the sector, or (in case of SSD) just calculating a memory address.

From my experience with (modern) Windows, "Initializing" just involves owerwriting couple of sectors with partition table. From this point owward, you'll see your drive as empty, ready to take partitions, in the same way as "fdisk /dev/sda" works on them. And "formatting a volume" is just writing the skeleton of an empty file system, being it FAT, NTFS etc - again, nothing different than "mkfs".

 

[rchamlen]

I'd just say that your Unix analogy is about 25-30 years outdated (unless you think about floppy / ZIP drives, where it might be still relevant). OS does not define physical sector boundaries, writes markers etc - all this is done by the disk controller. All the OS sees is CHS, and it is up to the disk controller to map CHS addresses into physical head movement / selection / waiting for the sector, or (in case of SSD) just calculating a memory address.

From my experience with (modern) Windows, "Initializing" just involves owerwriting couple of sectors with partition table. From this point owward, you'll see your drive as empty, ready to take partitions, in the same way as "fdisk /dev/sda" works on them. And "formatting a volume" is just writing the skeleton of an empty file system, being it FAT, NTFS etc - again, nothing different than "mkfs".

I only brought up the mention of sectors for the purpose of historical record .... if you look again at my post I do mention that later the formatting only became demarking the blocks - which physically spiral out from the inside of the disk (so I am aware of the difference). Looks like you are confirming my belief that the "initialization" is really just writing the partition table....

It is still possible to do a low level format. If it is as you mention that the block locations are taken care of by the disk controller (I might well be out of date - though those early disks still had controllers!) .... then just what is a modern "low level format" doing???

 

[rchamlen]

No .... wait ..... on a spinning disk there is no way the disk controller will be able to locate the circumferential location of block unless there are physical "markings" on the disk surface. Radial distance maybe ..... but not circumferential (rotational) location. So there must still be some lowest level formatting markers on the disk surface (Im talking HDD's here, not SSD's).

And yes .... you are right in the 25-30 year timeframe (and yes, I have just dated myself!) .... but on the other hand .... down the years a LOT of how things are done has just been inherited.....

 

[rchamlen]

I'd just say that your Unix analogy is about 25-30 years outdated (unless you think about floppy / ZIP drives, where it might be still relevant). OS does not define physical sector boundaries, writes markers etc - all this is done by the disk controller. All the OS sees is CHS, and it is up to the disk controller to map CHS addresses into physical head movement / selection / waiting for the sector, or (in case of SSD) just calculating a memory address.

From my experience with (modern) Windows, "Initializing" just involves owerwriting couple of sectors with partition table. From this point owward, you'll see your drive as empty, ready to take partitions, in the same way as "fdisk /dev/sda" works on them. And "formatting a volume" is just writing the skeleton of an empty file system, being it FAT, NTFS etc - again, nothing different than "mkfs".

I'm re-reading this in the morning of a new day .... and think I get t he comment you are trying to make. My analogy with Unix was not meant to indicate that the OS itself was responsible for controlling the head on the drive (that has always been done by the drive's controller in Unix machines) ..... I was only meaning to communicate that the sequence and what was being done was very clear. "formatting" was always the very low level process of writing demarkations on the drive surface. This was always followed by writing the partition table (similar to what the Linux "pvcreate" command does - but back then you would need at that point to decide whether the disk contained a boot sector - and yes it was called a boot sector), after which the disk could then be partitioned. To my memory this was not referred to as "initializing" - though it has been a while, and I might be remembering incorrectly (even though I did it many, many times.....).

so anyway .... based on your comment, it does sound like Windows' "initializing" does indeed just create (or overwrite) the partition table. Thanks to all for the clarification

 

[fzabkar]

In an MBR partitioned drive, the partition table lives in LBA 0. It consists of 4 entries which define the location and size of each partitioned section of the drive. In a GPT partitioned drive, the partition metadata is spread over sectors 0 to 33. Each entry occupies 128 bytes. A copy is also stored at the end of the drive.

Each partition consists of a boot sector and a file system. These metadata are laid down when the partition is formatted.

If the drive has no valid partition table, Windows reports it as "uninitialized".

If a partition has no recognisable boot sector, Windows warns that it needs to be formatted.

If the file system's metadata are severely damaged or unrecognisable, Windows warns that the partition is RAW.

As for low level formatting, the following article explains it all:

https://hddscan.com/doc/HDD_Tracks_and_Zones.html

A servo track writer is used to write embedded servo information to a blank disk. Each data sector is tested during manufacturing. Any defective sector is added to the Primary Defect List (P-List or slip list), and this list is stored in a reserved, hidden firmware area on the platters (System Area, SA). The heads and media are then tested and calibrated to find their optimum operating parameters. These parameters are also stored in the SA or in the ROM on the PCB. Because these parameters can vary significantly, it is often possible to fit more sectors on one surface than another. The zones are then recorded at different bits per inch (BPI) or tracks per inch (TPI) settings.

Therefore, it no longer makes sense to visualise the surfaces as consisting of concentric cylinders with a fixed number of sectors per track. Instead the drive keeps track of the relationship between LBAs and physical sectors by way of a "translator". This is a firmware module which lives in the SA. The SA contains many additional modules including SMART and the G-List (Grown defect list).

 

[hotaru.hino]

Documentation: https://docs.microsoft.com/en-us/windows-server/storage/disk-management/initialize-new-disks

When Windows is "initializing" a disk, it's setting it up to use either MBR or GPT style partitions. From there, you can create a new partition.