First of all, FX stock coolers are barely enough to handle their stock clocks/voltages. Upgrade to a good CPU cooler, I'd recommend a 120mm rad or 240mm rad if you want to overclock it past 4.8Ghz. Another note is what motherboard are you using? You need a decent motherboard before I recommend overclocking. With FX, given good cooling and motherboard VRM. They can reach 4.8Ghz-5Ghz without much of a problem usually. For voltage I'd recommend not going higher than 1.55V for a 24/7 OC, FX chips can take 1.6v and be fine but heat will be an issue.
Now to overclock you want to raise the multiplier by one at a time, stock would be 18 for your chip. So put it to 18.5 (3.7Ghz) then run Prime 95 and make sure no errors come up, for 1 hour to confirm it's stable. If it's stable then rinse and repeat, if not raise CPU Core Voltage by 125mv (0.0125V) and try again. Make sure to monitor temps and voltages while testing, I recommend using HWInfo and making sure CPU temps stay lower than 62C to be safe but 65C and under is still safe... ish.
Remember though, do not use the stock cooler when overclocking! You can get yourself way better CPU coolers for very little money these days.