Mr Erik :
I'm just confusing myself now. Some tests I've run in this order
1. Took another computer and plugged it in to the place where I've had my tower (can hear the interference on the headphones)
2. Hooked my tower up to my TV via HDMI (no sound problems through the TV but I can hear interference on my headphones)
3. Hooked the headphones up to the other computer I tested (Yep, interference after I close a game and turn up the volume on the headphones)
4. Hoping that I isolated the problem to be my headphones, I set my tower back up and don't plug them in (speakers still give off interference when I start a couple of games)
The largest lead I have again is that when I start Audio Surf from Steam, I can get a high pitched buzzing or even what you would call a whine coming from the speakers. However this same whine can be heard from what seems to be the graphics card (I took a paper towel roll to try and isolate the sound and it seems to be the 560ti that is making this noise.
1) It is possible that your other computer also has some interference issues. A LOT of onboard audio devices exhibit this interference pattern. Newer boards have a lot less of this, but all my older boards when combined with headphones leaks some form of interference.
2) Your TV probably has a low resolution sound system, so you can't hear it, or the sound is internally filtered (as in not full range coming from the speakers).
3) When you close a game, I suspect the GPU is clocking down, and this introduces high frequency harmonics in the power lines and on the GPU (spikes) when the GPU suddenly reduces its power consumption. But with proper motherboard design and layout, this shouldn't be an issue, at least not anymore...
4) That whine you hear on the graphics card is likely just fan or inductor coil whine.
It seems, at this point, if you want to get rid of the interference, you should use an external sound card of some sort... The internal looks like it has the ground channel heavily contaminated.