You're fortunate here to have a DDR3 capable board so you do have an upgrade path with RAM.
I'm not sure how intensive Shadow of Mordor is, but I can tell you that a Q6600 is more than capable of playing back 1080p60 video. These go for about $10-$15 on eBay and can typically overclock somewhere around 3.0GHz, putting them on par with a QX6850 or so. However, the Q6600 is a 65nm chip and your board would work with 45nm chips like the Q9400 (2.66GHz) or Q9500 (2.83GHz), though if you could get a good price on a Q9450 or Q9550 those are better since they have double the L2 cache (12MB vs 6MB). There's also the Q9650 (3.0GHz, 12MB L2) which would pretty much be the top end for your board.
So... you could upgrade the CPU to a quad core for under $20 and solve a good deal of general lag issues you have in 2D environments like stuttering with 1080p 60FPS video playback. Anything you do that's a little CPU intensive would basically double in performance or at least not hamstring other stuff so badly.
As far as games go, even a Q9650 would bottleneck most games and therefore in practice most GPUs are bottlenecked. You can still improve your performance, however. With a Core 2 Quad in place and another 4GB of DDR3 (maybe you can get a good price on used RAM?), it's entirely possible for you to handle those upgrades for under $40 and then look into a better GPU.
Anecdotally, I would assume that my 280X is bottlenecked but it's certainly playable in things like Bioshock. I rarely game so I don't care if something's bottlenecked or not, but when seeking out parts it's pointless to pay extra for more speed when you can't use it. So with that in mind, if we assume a 280X is overkill then a used 280X is going for about $175 because of Ethereum mining. Looking at the nVidia chips the GTX770 is essentially the same but seems to be landing around $100. Given your budget is about $100, we can look at the GTX760 which seems to be in the $50-$80 range. Compared to the Radeon 5670, the GTX760 is about 4x as powerful.
So... that's my suggestion. Find a used Core 2 Quad, 4GB DDR3, and a GTX760. For $100 you'll not find a better bang for your buck. More than double the CPU strength and quadruple the GPU power while also alleviating swap space hits due to the 4GB RAM.
Once you have these parts your system is officially maxed out. The board can't support a better CPU and the CPU can't support a better GPU so there's literally nothing for you to upgrade. To upgrade it you'd need to replace the motherboard, CPU, and RAM. The upside here is that the GTX760 can get more out of better CPUs, so this sort of upgrade would improve game performance as well as general computing performance.
-------
If you really only cared about video playback, dropping in a $15 quad core CPU would solve it.