Are you familiar with how to get in BIOS?
You have an unlocked chip and Z series motherboard. The 8600K is quite easy to put a very mild OC on without having to change much.
I would ask if you have a half decent cooler. Is your case getting good airflow and it's vents filters clean....if currently so, leave your side panel off for this experiment. If the filters/vents are dirty, clean them and you can choose to leave panel on or off.
You will have to check your specific manual for where these items are but Gigabyte make this stuff pretty straight forward.
Without changing the voltage, change your multiplier up to 46.
See if XMP will allow you to load a 26xx profile for your RAM
save changes and exit...no worries as if it is an issue at all it will just blue screen, you can boot back to BIOS and "default" in that same screen you save an exit from.
side note...do the multiplier increase first. Perhaps try x45, see if it boots, then x46. If it then boots fine move on to the XMP profile that way if you do blue screen or restart you know what caused it.
See if you don't see an improvement from that.
For more reference, with some other changes you should readily be able to OC that chip to 4.8 or maybe a bit better. I would suggest doing some reading about the power states and load leveling, etc. Just stick to a simple one first and see what happens. Watch your temps.