AMD A8 6600k Overclock

depends if you'll actually be doing anything that would actually make use of your overclock which is essentially setting your CPU to a higher speed to allow more demanding programs to run faster on your pc. OCing is typically done using the BIOS on your motherboard by tweaking the multipliers, voltages etc

When OCing you should always run a stability stress test to make sure that your CPU is stable running at those speeds.

Have a look at this here its an old guide but is still pretty valid in what you should do http://www.pcworld.com/article/198882/overclocking_for_newbies.html

But before you OC what motherboard and cooler are you using with your CPU
 
I have the A68HM-plus Motherboard with its default cooler

You can overclock on that board but you'll want to install a decent down blowing aftermarket cooler such as the GeminII M4. The stock AMD ones are fine at stock speeds but are very little use for overclocking and the heat it can produce. The reason for a down blowing cooler is because the VRM on that board doesn't have its own heat sync so may overheat causing throttling.

But even if you don't get a down blowing cooler you I would advise getting an after market cooler of some sort anyway :) the hyper 212 evo is one of the best value coolers around.
 
you'd actually be surprised as to what the stock heatsink and fan can handle. Although it's not recommended and you risk damage, it all depends on how much more power you want.

I have my A8 6600k on stock heatsink/fan running from the stock 3.9GHz to 4.2GHZ @1.275V.

I'm not sure if yours is the same, but mine was stock at 1.399V, and even at high overclock speeds you shouldn't need more than 1.45

I definitely recommend turning your voltage down before doing anything.

Go down one setting, boot, run stress test (I use prime95) and repeat until you get errors/or it doesn't boot.

I was able to get my voltage down to 1.25V, before starting to change my multiplyer.

As long as you measure your internal temps, and your thermal margins you should be fine