Switch computer off.
Remove the power button connector from the motherboard (exposing the two power pins on the motherboard).
Short the two power pins on the motherboard using a small metal object (small screwdriver blade, or paperclip etc).
If the computer boots/runs OK, then the case power button may be faulty.
If the computer still doesn't boot/run right away, then there may be a problem with the motherboard and/or PSU.
Try another known working PSU, and/or try the original PSU in another computer.
If the computer boots OK with the replacement PSU, and/or with the original PSU in another computer and that computer also has trouble booting, then the original PSU is starting to fail.
If the computer still has difficulty...