It'll work OK, better if you overclock. Since you'd need to buy a CPU cooler for that, you'd be better off getting an i3 for roughly the same price.
No, never cheap out on the power supply. That's the one thing that could make your whole system go up in smoke. However, right now there's a super-cheap deal on an XFX unit that's one of the highest-quality around for only $26:
http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=125449&CatId=5431
That's a normally $60-$70 power supply for $26 thanks to all the discounts and rebates. Get that one.