Did you reboot after unplugging the external drive?
We are most likely looking at a driver conflict , we just need to figure out where.
One other possibility is a program/ system utility that is starting on boot. Even though the actual program isn't running up CPU usage, it might be causing the other programs to freak out.
Go to Start --> Run --> type msconfig
When the window pops up, in the general tab, click selective startup, and deselect load startup items.
Reboot. A lot of things will probably not work correctly, but see if it fixes the CPU usage issue. If it does, then you can run msconfig again, click on normal startup, then go to the startup tab and deselect all the items, and enable them one by one (reboot inbetween) to see where the problem lies. This will take some time, but if disabling the startup items really solves it you'll definitely want to spend the time figuring out which item is causing this.
You can also click on the diagnostic startup in the general tab for msconfig, and that could give you different results on reboot.
Just to check, you're running XP Pro or Home?