I'm in a similar boat. I have a FX-6350 and a RX 570 so we're not that far off and I also want to upgrade soon. I recently overclocked my CPU from 3.9Ghz to 4.4Ghz, with thermals being the limiting factor. I've noticed quite a boost in performance from it. The thermals can get pretty hot at high speeds which would damage the CPU in the long run, but if you're upgrading soon it shouldn't matter.
In short, I'd suggest doing it. The FX processors aren't the best so if you're willing to trade of that extra bit of CPU lifespan for those precious frames it's a good idea.
As I say, I'm new to this too but I would recommend it completely.
Not really, the FX series was a pretty bad CPU line for gaming. You may see a small bump but pretty much any modern graphics card will be held back in modern games. I would suggest a Ryzen 3600 build since its pretty much the best bang for buck right now.