I am using a Asus ROG Strix B350-F Gaming Mobo . While gaming the temps stay below 75 degrees celciusThose temps are a little high for a daily driver. You will either need to lower the voltage, lower the OC, or get a better cooler. Unfortunately, it looks like you lost the silicon lottery at 3.75ghz is not a very aggressive overclock.
What boar are you using? If you are using a cheap mobo, that could also impact your overclocking stability.
THIS>The reason why the temps are lower when gaming is because the game does not stress the CPU as much as a CPU benchmark. So the CPU is used less in the game, so the temps are gonna be lower.
Be default the 2600 will boost up to 3.9 GHz for lightly threaded loads, and around 3.6-3.7 GHz for heavily threaded loads. Pretty sure manually setting it to 3.75 disables the auto boost, so that's not much of an OC, might even perform worse in some cases.