If there is noise while the PC is off, it would have to come from the 5VSB supply, which may (or may not) be normal.
Under low load, the 5VSB supply may operate in burst mode and what you are hearing is simply the 5VSB supply starting and stopping. As long as the output remains within specs, everything is fine.
Alternately, it could also be the 5VSB supply being unstable, hitting the high or low limit, shutting off, restarting, hitting a limit again, stopping, rinse and repeat, which would be bad.
To determine which one might be the case here, you would at the very least need a digital multimeter with min/max memory function and better than 1ms detection window. Ideally, you would use an oscilloscope to see exactly how clean (or...