1) Buy a Cooler Master Hyper 212 EVO to cool things:
http://pcpartpicker.com/part/cooler-master-cpu-cooler-rr212e20pkr2
As said don't overclock on stock.
2) Lots of guides around:
http://www.overclock.net/t/1240156/oc-fx-8120-bios-settings-for-a-stable-4500-mhz
"OCed my 8120 to 4100 MHz (x20.5 mult., Vcore 1.36), that's seems to be easy, but my issue is getting a stable 4300 or even 4500..."
i used amd overdrive and set speed to 4100 and voltage to 14625,when my cpu is at 3.1 it uses 1,4375V without overclocking his normal speed...
3) How much you can overclock depends on:
- CPU cooler (and room temp, case cooling...)
- Motherboard (voltage stability)
- silicon lottery for your CPU
4) FYI:
For gaming the FX-8120 will bottleneck any good graphics card, but keep in mind that many FUTURE GAMES will take much better advantage of your CPU due to multi-thread optimizations in DX12.