The fan on the PSU doesn't need to turn on because the system isn't even trying to do anything. The PSU has no need to turn on the fan.
As for the status LED, it probably needs a CPU to be installed and powered to have a status LED show. When my dad and I were building a PC we forgot to plug in the CPU and got no beeping sound or status LED, but after plugging the CPU in the status LED turned on.
The PSU-won't-turn-off-unless-I-unplug-it thing probably has to do with your motherboard not having a way to know if it should be powered off or on (because there is no power button, assuming you are debugging your PC outside of the case), thus meaning it might be on by default. I've never run into this issue.