I don't own a 3080 nor do I know what memory spiking on a GPU means so:
I suppose if your temperatures are fine, the system is stable, and you're getting the numbers a 3080 should be pushing, Is it really a problem? Maybe you can open task manager and see what's happening when it spikes but thats all the advice I have.