on the gtx 10xx series, I believe the fans will even turn off if low enough heat in the built in fan profile. You could set for a constant fan speed that is higher regardless of gpu usage. I do not know your usage when the fan is ramping up and down (desktop/gaming, etc).
Would check for airflow issues first: lack of sufficient fans/slow case fans, clogged dust filters, settings for your case fans if they are MB controlled. Does your case have room to breathe with open airflow, or do you have it tucked in somewhere partially enclosed? Do you have cabling inside the case tucked and routed out of the way for smooth airflow, or are cables in the way and disrupting smooth airflow?