Nice troubleshooting!
The question is whether the coil whine is inherent to the GPU or is the PSU doing something (or failing to do something) causing the GPU to whine.
That may be answerable by using another PSU or testing the GPU in another known working (not whining) system. Determine if the whine stays with the host PSU/computer or follows the GPU.
Lots to consider:
https://clevercreations.org/what-is-coil-whine-how-to-fix/
There are many similar links online. Some good, some not so much.
There is no rush, as I understand the circumstances, to apply some fix. The whine is certainly annoying but no harm in further investigation.
Make a list of possible solutions and start with the solutions that are as simple and as straight forward as possible. Maybe a bit of dampening foam.
Some changes to the case and accoustics. Alter the vibrations.
Try other changes: even if the changes are not what you necessarily prefer (e.g, any overclocking) at least try to discover what, if anything, reduces or stops the whine.
Try just one thing at a time and keep notes.
And there may be other ideas and suggestions offered. Coil whine is a common problem.