Are you turning off the PSU without permitting a normal Windows/computer shutdown?
Is it just one static "snap" or brief "crackle" or an ongoing noise?
Update your post to include full system hardware specs and OS.
PSU: make, model, wattage, age, condition?