Have you downloaded or looked in the manual to see if that light is a fault indicator?
I have never "jump started" a PSU while connected to the components. I have no idea what damage, if any, could be done doing so.
What I am asking is, if you trace your case power button back to the motherboard it's sitting on two pins. If you have everything plugged in and ready to be powered on, and short those two pins, it's the same as hitting a power button. This eliminates the possibility that the case power button or the wire back and forth are bad.
If it doesn't power on in that method I would suggest that something is damaged or incompatible in some way. Real shame with that (even older) chip and mobo.