Depends on the game really. On most FPS games you'll be fine,but in some games that benefit from a quad core CPU(like SC2,BC2 etc.) it will hold your card back(but again,you should still get a decent FPS)
It wont not noticeably anyway. As Maziar said games that want more cores will do better with a quad core chip but i dont see it as a need to upgrade situation.