Safely Remove Hardware

still_life

Distinguished
Nov 4, 2006
204
0
18,680
When using a USB memory pen, is it necessary to use the windows "safely remove hardware" option or can you just yank it out without fear of anything bad happening???

Thanks
 

randomizer

Champion
Moderator
It depends on whether you have enabled or disabled caching to the drive. Go to the device manager and assuming you have the drive plugged in, select it under disk drives and double-click on it. Then go to the policies tab in the newly opened window. If "optimize for quick removal" is checked, then you can just pull it out, if "optimize for performance" is checked, you need to use the safely remove hardware thing.

EDIT: I always just pull it out and have never lost data.
 
The "safely remove hardware" bit is used because removable drives like USB sticks and camera memory have a file system on them just like your hard drive does. Most of these drives are small (<= 32 GB) so they use FAT16 as the file system type. FAT16 is an older file system and writes to the disk are simply done on demand. If the power to the machine is cut or the drive is simply yanked out during a write, this will result in incomplete data and possibly a corrupted file system on the drive. The "Safely Remove Hardware" bit calls for all programs accessing the drive to complete whatever reads/writes they need to as the drive will be removed. When all I/O is complete, the OS removes the drive from its list of usable drives and then pops up the bubble telling you it's safe to remove it.

If you formatted your memory card/USB stick yourself and put a journaled file system like NTFS, ext3, XFS, ZFS, HFS+ or ReiserFS, then the removal is different. All writes to the drive are first written to a journal, which is a temporary space on the drive. Once the data is written to the journal, then the data in the journal is written to its final location on the drive, overwriting the original data. Yanking out the drive will cause the following things to happen:

1. Yanking the drive out before the data has been completely written to the journal causes the original data on the drive to remain safe. The changed version of your file that was being written to the journal is lost as the FS sees that there is an incomplete write to the journal and flushes the journal the next time the drive is accessed.

2. Yanking the drive out after the data is written to the journal but before it can be completely written to the file system results in the partially-modified file in the file system being replaced with the good copy on the journal the next time the device is accessed. The user would never know that the final write never completed.

3. Yanking the drive out before any writes to the journal occur or after the data from the journal was successfully copied to the file system cause no issues as the drive is idle.

Safely removing a disk with a journaled file system causes the same actions as with the non-journaled file system but will also cause the disk to make sure that all of the journaled data has been written to disk successfully. Journaled file systems are more modern and much more reliable than non-journaled file systems. They still should be removed safely, but the consequences for not doing so are basically that it takes a longer time to access the disk upon the next access (due to journal-file comparison and flushes.)

The reason that journaled file systems do not commonly appear on UBS sticks or camera cards is because of Microsoft. Most people run Windows computers and Windows computers can only recognize FAT16/FAT32 and NTFS file systems out of the box. ext3 support can be easily added via a driver, but the other file systems require some external program to access (ReiserFS) or are completely unreadable in Windows (XFS, ZFS, possibly HFS+ as well.) NTFS isn't used on removable devices because it is covered by Microsoft patents while the FAT file systems are direct descendants of the original DOS file systems and are in the public domain. Device makers would have to pay MS a royalty fee for every device shipped if it was pre-formatted with NTFS. So the device makers have a few options:

1. Ship the drive with FAT formatting for free even though it's sub-optimal.

2. Ship the drive with NTFS formatting and pay MS $0.25 (IIRC) per unit shipped.

3. Ship the drive unformatted for free and require the user to format it on their computer, where they can format NTFS for free. This axes out-of-the-box working ability.

4. Ship the drive with ext3 for free and require that the users download the Windows ext3 IFS driver for free. This also axes out-of-the-box working ability and renders the device unusable on computers where the user doesn't have admin access to install the ext3 driver (work computers, public terminals.)

Most manufacturers want easy-to-use and cheap rather than good so they ship FAT-formatted drives. I leave mine formatted as FAT as cameras won't support anything but FAT16 and since I don't run Windows, I can r/w to FAT but basically only read from NTFS (write support is VERY limited.) I'd simply format my USB stick ext3 and be done with it, but I use my drive a lot on university computers and I highly doubt that the techs would install the ext3 drivers for me.
 

adnoto

Distinguished
Feb 20, 2007
48
0
18,530
This is a great topic and info.

I have an OCZ 2GB Rally USB drive and the stinking thing will never allow itself to be "safely removed." IIRC it did allow this process the first or first few times I used it but in the 100+ times since it is a no go. I just yank the thing and, though I have never lost/corrupted any data to date, I worry that I will. Any thoughts? I can't say for sure but I certainly don't remember reformatting the drive.

I am going to check out cache issue randomizer mentioned next time I use it. Perhaps that is the issue.

.
 

cutthroat

Distinguished
Apr 13, 2004
429
0
18,780
:trophy: Great post MU_Engineer. :trophy:

I have a 4GB flash drive I use just for readyboost on Vista, can I format it NTFS or ext3? Would there be any performance gains by doing so? What is the benefit of being able to use ext3 in Windows (does the driver work with 64bit Vista?), would it allow me to use my ext3 formatted drives in Windows? (I have triple boot XP, Vista, Ubuntu 7.04).
 
cutthroat,

The benefits of being able to use ext3 in Windows mostly stems from the fact that all Linux machines have support for ext3. If you wish to dual-boot a Windows and Linux machine and have a shared My Documents/home directory, this had to be done in FAT16/32 in the past as that was the only FS that both could read reliably. Windows doesn't like FAT32 >= 32 GB much, so this limited the shared partition. ext3 is a much more robust FS than FAT16/FAT32 and would be much better for a shared partition on a dual-boot system. Passing an ext3 USB stick between suitably equipped Linux and Windows machines would be much better than one formatted with FAT.

There is no support for the ext2/ext3 IFS driver in any x86_64 Windows operating system and no official support in Vista yet. Perhaps you could use the 32-bit XP driver in 32-bit Vista, but I'd not know if that would work. The driver supports Windows Server 2003, so since Vista's kernel is based off of WS2003, perhaps it would work with the 32-bit version...maybe. Here is the web site with the ext2/ext3 Windows driver.
 

randomizer

Champion
Moderator
So how would you format a flash drive into NTFS then? In windows XP (32 bit) I can only format it into FAT or FAT32. For XP 32 bit would there be any reason to have it formatted into NTFS other than the reasons regarding removal of the drive?
 
Hmm. Maybe Windows won't let you format the drive as NTFS if it is under some certain size. The advantages of NTFS over FAT16/FAT32 is that it is generally more robust, regardless of the device is uncleanly removed or not. NTFS also supports UUIDs and ACLs, which could possibly lock you out of your own files :?
 

valis

Distinguished
Mar 24, 2005
380
0
18,780
When using a USB memory pen, is it necessary to use the windows "safely remove hardware" option or can you just yank it out without fear of anything bad happening???

Thanks

no, eventually you'll get corrupt data on the usb drive
 

Elektriksheep

Distinguished
Apr 10, 2008
8
0
18,510


Q1. What are the (potential) consequences of a corrupted file system on the drive - will you "just" lose all data on the drive, or will the drive become permanently unusable?

Q2. For a standard windows user, using a drive that has been formatted to NTFS, if the drive is removed 'unsafely': I understand that the changed version of the file that was being written to the journal can be lost, but by using NTFS do you completely prevent any potential file system corruption?

Thanks in advance.
 
G

Guest

Guest
In our company we use sd cards and usb pan drives for passing drawings around. we have lost loads of both types. they just stop working. I also had a 1gig usb drive at home that broke. I assume its all connected to not using the safely remove hardware tab.
My advice is always use it no matter what!!
 

Elektriksheep

Distinguished
Apr 10, 2008
8
0
18,510
In our company we use sd cards and usb pan drives for passing drawings around. we have lost loads of both types. they just stop working. I also had a 1gig usb drive at home that broke. I assume its all connected to not using the safely remove hardware tab.
My advice is always use it no matter what!!

Yeah, I know what you're saying, I pretty much always do, at least since reading this thread.

I have a Seagate Freeagent external hard drive, and sometimes it does not respond to 'safely remove hardware' even when all programs are closed (I checked task manager to make sure not related processes were still running), it just says something along the lines of "Device cannot be stopped right now, device is in use."

So I either shut down my laptop (f***king hassle) to make sure nothing gets damaged, or yank out the USB hoping that my laptop is lying to me with its "Devise is in use" comment. When I do the latter though, I am still concerned that I might get a corrupted file system and lose around 200gig of pretty valuable data.

If anyone in the know has a response to my previous comment it would be greatly appreciated.
 

jonathanfox5

Distinguished
Apr 6, 2009
1
0
18,510
Electricsheep, perhaps you have a windows explorer window open? I.e the standard window that lets you see files i.e the window that pops up when you click on (for example) 'my documents' (sorry for not being able to explain it very well).

I would be willing to bet that is where your problem lies. The annoying thing about it is that they are very easy to forget about / miss when you are looking for items that need closing. I advise that you close ALL of these since some folders are named the same e.g a folder named 'Pictures' could be absolutely anywhere - on a cd, on your internal hardrive, on a usb flash drive or on your external hardrive.

Probably won't be visiting this exact webpage again so I hope that I have set you off on the correct footing for others to help you.

Regards,

Jonathan
 

Elektriksheep

Distinguished
Apr 10, 2008
8
0
18,510



I don't know if you'll get this, but thanks for replying. Unfortunately windows explorer is the first thing I check, but even when all windows explorer windows are closed I still sometimes have this problem. It doesn't happen all the time, but it is very annoying when it does.
 

f00dl3

Distinguished
Sep 7, 2006
5
0
18,510
Another overlooked consequence that I didn't realize could happen until it happened to me - shorting the USB controller out. Believe it or not I actually fried a motherboard by removing the USB drive without Safely Removing Hardware. It can happen...
 

Elektriksheep

Distinguished
Apr 10, 2008
8
0
18,510


Wow. Anyone else heard of this?
 

rahul_does

Distinguished
Jun 21, 2009
3
0
18,510


Hello there. Try Ctrl+Shift+Esc to bring the task manager and "kill" explorer from processes tab. Go to file and run a New Task and type in Explorer. You shall get your normal desktop back AND would be able to "Safely remove HardWare." Works in my case!
 

shrimppuff

Distinguished
Jun 24, 2009
1
0
18,510
i have heard from a fairly reliable source that just yanking it out can actually make it lose data storage capacity. is there any truth to this?
 

stevemholmes

Distinguished
Dec 17, 2009
1
0
18,510
I too share the battle with the "safe to remove tab"

I can not shut down y external usb conected hard drive with the "safe to remove" option.
I am running windows 7, a new pc laptop, the usb hard drive is a "Seagate" and using the Sesgate Back Up Manager software.

I have checked the task maanager, made sure all related programs and devices have been stopped: however,
I still recieve the message, "the device is still in use by another program or device".

----Still Pulling the Pug----
 

LordNyson

Distinguished
Apr 3, 2010
1
0
18,510
Yeah, I was interested in this topic.. I had the device set to optimize for quick removal or whatever, unfortunately this setting did not carry over to the uni computers... when I did not safely remove it caused the portable HD to become an "Unknown Device" to all windows systems... so I opened up the HD and plugged the HD directly into my computer. No data corrupted... seems the adapter stopped working.. and now my head hurts because that doesn't make sense!