The only way to test if heat is the problem is to put the card back in the system.
Use a Gpu temp monitor, and see how hot the card gets after an hour of use playing a game or something, 55c to 65c should be fine with the card after say an hour of playing a game. But if the temp reports back as 90 c to 100 c, then you may be looking at a cooling problem. You can turn the fan speed up on the card manually with the nvidia software and see if it helps. The only other thing it could be is a badly fitted cooling solution on the card, a lack of thermal paste between the Gpu and the cooling plate of the heat sink cooling solution of the card. Check if it feels nice and tight without moving around.
But if its new, and your not happy return the card under RMA for a replacement, if you have the delivery sheet for it or a receipt, or any kind of purchase record.
The Thermal limit of the Gpu is about 130 c at max before meltdown, and damage may happen BTW. But if you are getting any type of things like colored dots or squares it likely means the card is overheating ok.