Linux depending on it's distribution can be installed on a USB Flash Drive, so I wouldn't see why it would fail on an external HDD. I know of several users in the Ubuntu community who do this to test new OS's for stability, or troubleshooting for other users. The testing community for Linux practices this quite frequently as a failsafe that's more stable than a virtual environment.
The process for doing this is as simple as one would expect. When you have an External HDD formatted, make sure it's been formatted by a Windows machine first. I used a Windows XP SP 3 for this, setting it as drive "U" so that it wouldn't conflict with the standard assignment of the installed HDD/SSD's.
Find the distro of Linux you like, I'd recommend Canonical's Ubuntu simply because it's an easy, user-friendly environment for a new user.
http://www.canonical.com
You need to download the DVD media for the subtype of Ubuntu you like, I.E. Kubuntu, Lubuntu, each work differently for different types of computers. The website will tell you all you need.
Burn the image (distro.ISO) and once done, put it in your CD Drive and enter the BIOS on boot, select boot from CD Drive. This will bring up the installation prompt. Make sure prior to this, the external HDD is connected to your computer.
It will then ask you, at some point which partition you would like to put this on. The Linux definition of your HDD will differ from Windows, and will not label the Drive Letter. Make sure you're aware of the free space on the HDD prior to entering this screen so you can identify which is your HDD when installing. Choose the one you want.
It will install to the HDD, and since Linux is an OpenSource OS, it won't have the same anchor Windows does.
That should work, if you have any more questions, the Canonical community can help you further.