If it's sort of a "bum-bum" noise from the speaker, this is just Windows making a signal that a device has been plugged in or unplugged. Is this the sound you hear?
When you eject the drive, do you properly unmount it first? In the system tray, probably in the section that pops up when you click the little arrow near the clock, there's an icon "safely remove drives." Make sure to click this and then click the removable HDD first before you unplug it. Removing a drive while it is still in use can corrupt the data on the drive.