You can't just use whatever cable you want. You need to buy a cable that's specifically for the EVGA BQ 750. That typically means contacting the manufacturer or buying aftermarket cables that are clearly labeled as having the pinout for the exact PSU in question.
You're lucky it didn't fit as there's a high chance that you would no longer own an RTX 2080 as a result.
I'm not sure what a "PCIE to VGA" is, so you're going to have to explain if this is something else you're talking about. PCIE is the slot that communicates between the motherboard and the graphics card. VGA are older display ports and if your RTX 2080 has a VGA port, it's a fake 2080; Nvidia has long since stopped including these ports on their GPUs.