Don't feel like "just pulling the plug" would cause such. Now, something along the lines of "I tripped over the plug and it caused the whole computer to jerk as the plug came out" to be a more likely causation (type thing).
So, the first thing I would do would be to unplug the machine, turn off the PSU switch. Open the side panel and reseat the GPU. I would also check to be sure some cable or connector didn't get jarred into the fan causing the noise.
Test it with the side panel off such that you can hear or see the issue more clearly.