Basically I have a home server sometimes remote access doesn’t want to work so I gotta go shut it down stick in a gpu in get a desplaiy do what ever shut it down and pull the gpu back out again
Don't get me wrong, you can transmit video over USB 3 or thunderbolt (USB-C), but you still need to generate the video stream and for that you will need a Graphics device on your system.
You can buy an external GPU, however it will be far more expensive than a simple GT730 or 1030.
https://www.makeuseof.com/tag/everything-need-know-external-gpu/