Using my Asus Mobo settings I get 4.3 Ghz at 1.25V with Evo coolermaster 220 runs max 53 degrees with all fans set to auto (again using the asus management)
My 4690k runs at 4.6ghz stable at 1.25v, 60c while gaming with a Cryorig H7 air cooler 32c idle temp. You should be able to get somewhere between 4.1 and 4.7 depending on chip lotto. It doesn't HAVE to be a water cooling system, there are plenty of air coolers that get identical temps to most of the AIO water coolers. But you will need some kind of aftermarket cooler.