The "safely remove hardware" bit is used because removable drives like USB sticks and camera memory have a file system on them just like your hard drive does. Most of these drives are small (<= 32 GB) so they use FAT16 as the file system type. FAT16 is an older file system and writes to the disk are simply done on demand. If the power to the machine is cut or the drive is simply yanked out during a write, this will result in incomplete data and possibly a corrupted file system on the drive. The "Safely Remove Hardware" bit calls for all programs accessing the drive to complete whatever reads/writes they need to as the drive will be removed. When all I/O is complete, the OS removes the drive from its list of usable drives and then pops up the bubble telling you it's safe to remove it.
If you formatted your memory card/USB stick yourself and put a journaled file system like NTFS, ext3, XFS, ZFS, HFS+ or ReiserFS, then the removal is different. All writes to the drive are first written to a journal, which is a temporary space on the drive. Once the data is written to the journal, then the data in the journal is written to its final location on the drive, overwriting the original data. Yanking out the drive will cause the following things to happen:
1. Yanking the drive out before the data has been completely written to the journal causes the original data on the drive to remain safe. The changed version of your file that was being written to the journal is lost as the FS sees that there is an incomplete write to the journal and flushes the journal the next time the drive is accessed.
2. Yanking the drive out after the data is written to the journal but before it can be completely written to the file system results in the partially-modified file in the file system being replaced with the good copy on the journal the next time the device is accessed. The user would never know that the final write never completed.
3. Yanking the drive out before any writes to the journal occur or after the data from the journal was successfully copied to the file system cause no issues as the drive is idle.
Safely removing a disk with a journaled file system causes the same actions as with the non-journaled file system but will also cause the disk to make sure that all of the journaled data has been written to disk successfully. Journaled file systems are more modern and much more reliable than non-journaled file systems. They still should be removed safely, but the consequences for not doing so are basically that it takes a longer time to access the disk upon the next access (due to journal-file comparison and flushes.)
The reason that journaled file systems do not commonly appear on UBS sticks or camera cards is because of Microsoft. Most people run Windows computers and Windows computers can only recognize FAT16/FAT32 and NTFS file systems out of the box. ext3 support can be easily added via a driver, but the other file systems require some external program to access (ReiserFS) or are completely unreadable in Windows (XFS, ZFS, possibly HFS+ as well.) NTFS isn't used on removable devices because it is covered by Microsoft patents while the FAT file systems are direct descendants of the original DOS file systems and are in the public domain. Device makers would have to pay MS a royalty fee for every device shipped if it was pre-formatted with NTFS. So the device makers have a few options:
1. Ship the drive with FAT formatting for free even though it's sub-optimal.
2. Ship the drive with NTFS formatting and pay MS $0.25 (IIRC) per unit shipped.
3. Ship the drive unformatted for free and require the user to format it on their computer, where they can format NTFS for free. This axes out-of-the-box working ability.
4. Ship the drive with ext3 for free and require that the users download the Windows ext3 IFS driver for free. This also axes out-of-the-box working ability and renders the device unusable on computers where the user doesn't have admin access to install the ext3 driver (work computers, public terminals.)
Most manufacturers want easy-to-use and cheap rather than good so they ship FAT-formatted drives. I leave mine formatted as FAT as cameras won't support anything but FAT16 and since I don't run Windows, I can r/w to FAT but basically only read from NTFS (write support is VERY limited.) I'd simply format my USB stick ext3 and be done with it, but I use my drive a lot on university computers and I highly doubt that the techs would install the ext3 drivers for me.