FPS drops/low | TF2/GMOD

[Shockbopz]

I had been playing tf2 and gmod on a shitty dell since 2010, and i got 60 fps on med with the dell. Now i recently built a new pc, and im getting low fps even on lowest settings. It'll stay on 300fps for like 10 seconds then it drops to 40/30 for like a good few minutes and comes back up again. I was playing casual/TTT. But, this ONLY happens in tf2. I can play overwatch an EPIC settings with some adjustments and get 70 fps constant. 300+ fps on cs:go ECT. All my temps are fine, it's not overheating or anything. I just can't seem to figure it out.
SPECS -
r7 370 4gb edition
amd fx-6300
8gb RAM
some gigabyte motherboard with a bunch of letters and number no-one cares about.

And yeah, pls help me.
im desperate.

 

[AMURR]

reinstall Gmod and see if that fixes it

 

[Shockbopz]

reinstall Gmod and see if that fixes it

Doesn't seem to have fixed it. I did some tests and my gpu maxes at around 35% and all 6 cores max at 70%, I would think they should be 100% all the time for the best performance?
UPDATE: It seems it only happens in multiplayer servers, if i go into single player i get constant 300 fps.