I'm thinking this might be the power supply. Are there any other components in a PC that electricity is running through when the power is off that could be making the noise? I know that electricity runs through the motherboard when the PC is turned off, at least to the point of the motherboard's LED, but I don't know if there are any components in that path on the motherboard which could be making the noise.
When the computer is unplugged or turned off via the power supply's rocker switch, the noise continues for a couple/few seconds and then you can hear it quickly wind down and stop.
When the computer is unplugged or turned off via the power supply's rocker switch, the noise continues for a couple/few seconds and then you can hear it quickly wind down and stop.