Are you running multiple high refresh rate monitors? I know on my RX480 8GB the memory will stay at 2000mhz when I plug a second monitor in. From what I had read in the past, this is normal.. for AMD. There's only 2 levels for this card's memory. 300mhz and 2000mhz. Even with 1x 144hz monitor, it bounces around here and there from 300mhz to 2000mhz. Since they are designed that way according to AMD, it doesn't worry me, it just kinda peeves me that when my memory is at 300mhz, the card idles at 7-8 watts, but when the memory is at 2000mhz, it idles between 20-25 watts.
Not sure if Nvidia is the same way.