Sitting on my desk right now is a Sony Playstation 3d Display (60hz on PC) and a Benq xl2420t (120hz). 120hz does make a difference under the right conditions. The single largest difference I've noticed between these two displays is that the Sony has serious screen tearing that needs to be addressed with Vsync while the Benq seems to not tear no matter how many FPS I'm getting. Also, I really don't like running vsync so I really like the Benq.
I'm not saying that all 60hz panels tear like mad, just that the one on my desk does. Tearing drives me nuts.
If your computer can render at 120FPS and your monitor can refresh at 120hz games become liquid smooth, it's really quite remarkable. If you gamed in the 90's or on CRTs you can actually get some crazy deja vu at times. Especially with Valve releasing new CS versions...
To answer your question then, the monitor you choose can make a big difference in your experience, but finding the correct monitor for your rig depends on a lot of factors. If your computer is going to be used primarily for gaming and you're willing to pay extra for a TN panel that refreshes more frequently (a "gaming" montior), IMO, it's worth it. If your computer is going to be more "general use" and you're not going to throw a lot of money at GPU(s) then you're probably better off with the IPS panel you already have.
I've gamed on IPS panels, it's fine. Colors are nice i guess. 120hz has made a snob out of me however; people react differently to the effect.