As long as it uses PCIEx16 it should be compatible. The power supply plugin may not fit, but easily buy an adapter for that. Sometimes they even come with these.
I say go with the GTX 750ti, of course the 10x series of NVidia is better but they require more power and more cash. AMD is a good cheap choice but they tend to be slightly more power hungry.
i.e. GTX 750ti @ 64 watts vs R7 360 @100 watts. (The RX 460 is designed for <70 watts so it's about the same)
RX looks to be about $10-$20 more expensive but the benchmark is higher slightly.