What cooler do you have on it? is it reference or after-market?
Do you have sufficient airflow in your case? I had a problem with my MSI gaming 980ti that was solved after i reversed my airflow to flow towards the card in the case rather than away from it - dropped my idle temp my 10c-15c.