The 970 has less effective RAM 3.5gb (512meg is like 32 bit which does work in games well) but is a bit faster even at stock speeds. The speed difference is not really that significant though and the 480 has 4 to 8 gigs of RAM. The upcoming 1060 is rumoured to have 980 performance and 3 or 6 gb of RAM, and might be worth checking out before you make the jump. There are some nice aftermarket cooler 970s that are well overclocked that would make a difference, might also be worth checking that out.
Hope this helps,
Gabor