Just for reference:
I currently have my Ryzen 5 2600 OC'd to 3.9 Ghz @ 1.275 V with my 3200 MHz RAM OC'd to 3400 MHz on an Asus ROG Strix B450-F Gaming MB.
I was able to get a stable 4 GHz with 1.33 V and got as high as 4.075 GHz at 1.36 V.
While 4 GHz and 4.075 GHz did add a few points/fps in benchmarks,
I decided to stay at 3.9 GHz because I leave my PC on 24/7 for streaming.
I do use the Ryzen Balanced Power Plan too though so my CPU does downclock to 1.55 GHz when idle or under light loads.
I also do NOT use the stock cooler.
I have an Arctic 33 Esports Edition which keeps my CPU pretty cool, idles @ 28-30C and maxes out around 55C after a few hours of gaming/heavy load.