Update your post to include full system hardware specs and OS.
Overall, you are probably correct.
All sorts of forces are in play and as the fans speed up the GPU may be twisting or bending a bit causing a fan blade (or blades) to come in contact with the housing. Especially if the GPU is on the "flimsy" side of things. Made of thinner metal and not strong enough to resist even minor bending or twisting.
Fan blades could be out of balance and start a wobble that leads to contact. Or could be the fan bearings - especially if the fan was force spun by canned air during cleaning.
Try to narrow down the source location of the noise. A cardboard or paper tube can be used as a listening device to listen around the fans and their...