Ok, so you just PROBABLY answered your own question. Bottom line is, with a cheap power supply, which 90% of the time IS the issue anyhow, it's pretty much impossible to figure out what else MIGHT be wrong with the build because none of your hardware can function correctly if the power supply is not functioning correctly.
A bad power supply can look like pretty much every other hardware component has failed or is failing because they ALL rely on the power supply. Power supply is no bueno, everything is no bueno.
That being said, I would STILL first try running the DDU as outlined in my tutorial that I linked to above. The AMD utility isn't worth a crap. It doesn't remove bad registry entries or many of the files and folders that often cause problems. The Display driver uninstaller (DDU) removes ALL of the commonly problematic registry entries and folders/files/drivers that cause issues. So follow the instructions, run the DDU, do a clean install of the most recent AMD drivers for your card and then see what happens.
Also, if you have not already you might want to find and install the latest AMD chipset drivers off the AMD website, AFTER you run the DDU and reinstall the GPU card drivers.
Since your CPU is so old, it's possible that there is simply a resource incompatibility due to the age of the hardware OR a problem with your chipset.
If none of that help then I'd probably replace the PSU, actually, I'd do that anyhow. You REALLY need a better PSU, I hate to say it since you apparently just got that one, but there is is anyhow.
And, if it has been a long time, or never, since you did a clean install of the Windows operating system, that's probably not a bad idea if nothing else seems to help. Something is triggering that code and if none of this works it may just be a bad card.