The mains connector socket on the back of a new PSU in combination with a new mains power cable can often be a tight fit first time, so make sure you've pushed it in far enough, otherwise the PSU will not get any power.
What is the "ATX" cable? Are you talking about the 24pin to the motherboard? Check all power connections and reseat ram and GPU as they may have come loose in shipping. If you are plugging the 24 pin in with power on it that is a good way to mess up your board.
Yeah, the 20+4 connector is what I meant, it's labeled as an ATX socket. I tried swapping the RAM slots, but haven't tried fully removing the GPU yet. I wasn't aware that could mess up the motherboard though, thanks for the warning.