Hey I'm experiencing the same issue with a 3600 and I think a combo of these solutions might work but I don't know enough.
https://www.userbenchmark.com/UserRun/29789867 for specs.
I'm idling at 65C, when I load NFS: Heat I hit the thermal limit and it shut off last night which is what got me investigating.
I'm running default everything, I've changed Windows Power Saver to 10% instead of 99% min, I cleaned and reapplied thermal paste, I'm using Ryzen Master for the temps, and nothing has gotten it any lower. I found this thread and I tried undervolting by 0.05, and it went down about 5 deg. Then I tried the BIOS Throttle. That being said, I don't know which option is better. I think 0.1 might be enough to get it to idle in the 40s instead of 50s, but I don't know if setting that permanently is better than just having it throttle at 80 degrees? The issue with throttling at 80 is that in NFS: Heat which I just tried, I was able to successfully play but at what cost? It was sitting at 80 deg the entire time and I have no idea what it's doing in order to keep me there since I know it would exceed 100C+ if it could.