First of all, FX stock coolers are barely enough to handle their stock clocks/voltages. Upgrade to a good CPU cooler, I'd recommend a 120mm rad or 240mm rad if you want to overclock it past 4.8Ghz. Another note is what motherboard are you using? You need a decent motherboard before I recommend overclocking. With FX, given good cooling and motherboard VRM. They can reach 4.8Ghz-5Ghz without much of a problem usually. For voltage I'd recommend not going higher than 1.55V for a 24/7 OC, FX chips can take 1.6v and be fine but heat will be an issue.
Now to overclock you want to raise the multiplier by one at a time, stock would be 18 for your chip. So put it to 18.5 (3.7Ghz) then run Prime 95 and make sure no errors come up, for 1 hour to...