The only reason to even consider changing cases is if the gpus physically do not fit. There's different versions of the 290x, 970, 980 etc and they have different lengths. All you'll need is either from the back of the case to front (interior space) or to hdd bays, fans or any other limitation on gpu length. Sapphire built a monster 3x fan R9 290x and Evga built a tiny single fan 970. So you have options.
It's an HP. That's proprietary connections in that case, so changing out the motherboard is a huge deal and not recommended. Going that far, you'd be better off just buying a new mobo, case, ram as that old i5 2400 is underpowered for a card like a 290x. Bur buying a whole new pc based around a gpu decision isn't the best idea ever, and not really needed if otherwise you are OK with what you have.
Psu is an absolute must with that card. Generally a good 650w (good, not junk) psu is recommended although ppl make do with less. A lot depends on the actual model, the monster Sapphire Tri-X is a huge power hog, dual 8 pin pcie, while reference designs make do with 1x 8pin & 1x 6 pin.
Personally, with your cpu, case, monitor, budget etc you'd be better off with a gtx 970/980. A good 550w psu is plenty for any model, most models are smaller, less power hungry and run far cooler while still topping out 1080p @60Hz without issue.
My OC 970 and i7-3770K (4.6GHz OC) runs quite happily on a Evga G2 550w and has for the last 6 years or so. There isn't a game I own that gets less than 60fps.