You might actually have enough room for the 7750 in the stock case. It is a pretty short card. There is a pic of the interior of your case on this page, select the last pic:
http://reviews.cnet.com/desktops/hp-pavilion-slimline-s5120y/4505-3118_7-33698025.html
There are two wires that look like they are in the way and the RAM is pretty close but it looks like you have enough room for the card you want.
EDIT: those two wires appear to be zip-tied. If you simply cut the zip-tie they should have enough slack to clear a GPU. And even if you get a new case, it won't change the location of the RAM. I suggest you measure the space you have before buying the GPU.
As for the PSU, your PC has a 220w (http://reviews.cnet.com/desktops/hp-pavilion-slimline-s5120y/4507-3118_7-33698025.html) so it will definitely need to be uprgaded. A 430w CX430 should be enough:
http://www.newegg.com/Product/Product.aspx?Item=N82E16817139026 it is currently $25 on newegg after a rebate.