you'd actually be surprised as to what the stock heatsink and fan can handle. Although it's not recommended and you risk damage, it all depends on how much more power you want.
I have my A8 6600k on stock heatsink/fan running from the stock 3.9GHz to 4.2GHZ @1.275V.
I'm not sure if yours is the same, but mine was stock at 1.399V, and even at high overclock speeds you shouldn't need more than 1.45
I definitely recommend turning your voltage down before doing anything.
Go down one setting, boot, run stress test (I use prime95) and repeat until you get errors/or it doesn't boot.
I was able to get my voltage down to 1.25V, before starting to change my multiplyer.
As long as you measure your internal temps, and your thermal margins you should be fine