The light on the GPU is not full assurance that the computer has power.
PSU's provide power via three different voltages: 3 volts, 5 volts, and 12 volts.
So if one voltage is not working then the computer may not start.
However, if second PSU did not work then there could be a case switch problem or maybe a faulty motherboard connector.
Did you use power cables from the old/original PSU to connect the new/test PSU? Could be a bad cable.
Do you have a multi-meter and know how to use it? Or know someone who does?
FYI:
https://www.lifewire.com/how-to-manually-test-a-power-supply-with-a-multimeter-2626158
Not a full test because the PSU is not under load. Any voltages out of tolerance make the PSU suspect.
The multi-meter can also be used to test the case switch.