This is a very very complicated question. There are so many places all I can do is give you a taste of some of it. One could spend years on the topic. Forgive me if I provide too much or too little information (this is only to give an idea of how difficult it is to answer). What I'm about to tell you is relevant, but might just bore you to death.
A shorter answer is that the power supply might be picking up the change in load from the GPU. More VRMs with more phases helps sometimes, but it also just changes the frequencies which the audio might pick up. You'll notice that many motherboards for overclocking enthusiasts feature more VRM power phases. Or the power supply itself, if it is capable of providing more power than is used (meaning overkill on PSU current output capability), might reduce the whine (the frequency of noise on the rail would be the same, but the amplitude would go down for a more capable supply).
It might also be some trace on the motherboard being too close to something in audio traces. Every trace also has to have a return path for the electric current. The design to eliminate noise on analog traces typically requires putting the analog content on a different part of the motherboard, but there is almost always at least some EM effect when sharing a motherboard. The more compact the design the more difficult this is. Regardless of the design some frequencies will provide "cross talk" when others do not. A different motherboard might help.
Shielding around cables and different parts of the system can also help. Some cables have just a little bit of shield braid, others of higher quality are quad shield with multiple shield braids over each other for better shielding.
Now for the part which will bore most people...read with caution. Have your emergency coffee rations handy. Read only for the trivia value if you want to know why it gets so complicated.
Whenever you have a pure sine wave noise issues are at their smallest. Think of two batteries, and if they are 1.5 V each, and in series, then they add to 3 V. If one of the batteries is inverted, then instead of adding, they subtract and become 0 V. The sine wave can be thought of as a perfect AC electric alternator spinning without wobble.
Now if you have two AC alternators (any sine wave source), and you put them in series, the output voltage gets more complicated. If the two alternators have a 1.5 V peak (3 V peak-to-peak) voltage, and if they are in phase with each other (same frequency and peaks and valleys at the same time), then they can add to become a 3 V peak voltage sine wave (6 V peak-to-peak). This just becomes a higher voltage of the same waveform.
If those alternators are 100% out of phase (peak of one at the moment of the inverse peak of the other, but same frequency), you get 0 V.
But computers don't use sine waves. They are digital 1s and 0s. These are not actually perfect square waves, but they are fairly close to this with rise times in the realm of nanoseconds. There is a way to look at any signal and determine what mix of sine waves make up the more complicated wave. The math provides a series of sine (actually, sine and cosine) waves, that when added in the proper proportions (anyone remember that in the Tremors movie?
), can be put together to create the more complicated wave form. Finding the mix of those sine waves is a Fourier decomposition, and mixing them together to create a new signal is Fourier composition.
There are frequencies which are considered odd or even "harmonics". Signals which are an exact even multiple of the original signal are even harmonics, and those which are an odd multiple are odd harmonics. When speaking of harmonics, the original signal is the primary signal, and one typically gets different harmonics due to reflections in cables and traces. The result of odd and even harmonics is that they tend to be of a lower amplitude than the primary signal. The higher the order of the harmonic, the lower its amplitude if it is a result of reflections.
As it turns out a "perfect" square wave is the infinite sum of the odd harmonics. If you have a signal that is 200 Hz (imagine a monitor scan rate), then a square wave really has components that include 3*200Hz (a 1*200Hz is the primary signal), 5*200Hz, 7*200Hz, 9*200Hz, and so on, with decreasing amplitudes. A perfect square wave is also impossible because it requires infinite energy to have an instant rise from 0 V to some positive voltage. One would create a black hole from the energy density before reaching a perfect square wave. Even so, the odd harmonics, although not being infinite, make a really great noise source for unshielded systems. If that square wave is sometimes not all perfect 1 and 0 transitions, then there are more components. The primary can in fact end up being considered multiple primary signals that are not odd or even multiples of each other. The harmonics of the different primaries then add and subtract to provide more harmonics (but they are neither even nor odd harmonics, they are harmonic distortion). All of which can provide noise at audio frequencies. Changing the video signal by moving the mouse changes the primary frequencies. There are many. Changing the video signal by moving the mouse changes odd and even harmonics, along with mixing of other harmonics. It is a mess!!
EM shielding can help because it changes the return current path to not include the wiring for the analog audio. However, if the analog and digital signals share a return path (ground), then they will still get mixing. One can minimize this, but there are tradeoffs (like larger motherboards that won't fit in the case, and larger traces with greater spacing). It takes a really ingenious design and software analysis to make everything fit together without the audio getting some noise. The slightest change in layout of the PCB, layer stacking, and many other details means some motherboards will just suck under some loads, and not have noticeable noise in other loads. Minimizing amplitude changes in the power delivery is one of the places where motherboards can compete (remember...same signal issues, but lower amplitudes if power delivery is more efficient and if the power supply itself has lower voltage drop as the components draw power).