Yeah, at this point I'd just keep the 320MB. Unless you could sell it easily and get the GTX.
From now on (regretting myself not going GTX) I plan on getting as beefy a videocard as possible as upgrading a computer isn't EVERY year anymore like in the late '90s and game development is geared towards the consoles.
The videocard does make you a much better player in FPS games. I used to be a mediocre player because I ALWAYS had a low-end card. With the 8800GTS 640 in UT3 all you hear is "headshot headshot headshot HEADHUNTER" because when the game is silky smooth it makes it alot easier.
So, if everyone is stuck waiting to upgrade when the consoles start "breaking new ground" it may be a LONG time until the PS4 and xBox "720" are out, you might as well get a heavy hitting card and enjoy it as the cost-per-year of use goes WAY down.
I had a buddy in the late 90's who ALWAYS budgeted for a NEW system every year with a new videocard every 6 months. But then again, games were easy to come by from friends, so I don't think he paid 60$ for a game EVER.
The consoles, and the quality of games we are now at, things are becoming a better investment.
I went GTS expecting a WHOLE new system with PCI-2.0, but gauging how console-centric everything is my Opty 165 is going to be with me for quite a while. Also note that Vista is a flop, so the pressure to upgrade is much less.
Anyway, I personally don't mind buying videocards anymore. I played some FPS with a gamepad when a friend got the xBox 360, and all these PC expenses are worth it. With WASD and a Mouse, you have more of a "feeling" that you are there that I don't get with a controller, but that is just me.
So my point is if you WANT a GTX, even if you have to stick the 320 in the closet and "lose" the money on it, you will be using the GTX probably until the PS4 is out, which is a long time.
Don't feel bad Bro, women throw new shoes away because they "don't like them anymore"...