You want an ATX power supply and buy a case that fits an ATX power supply. 90% of all case support an ATX PSU and 90% of PSU are ATX. It is hard to mess it up. Yes, you do need to worry about a GPU fitting in the case. Some cases this is not a problem, but smaller cases it is. Every major brand of case manufacturer will say what size GPU is supported. Each GPU also list the size of the GPU. So the case may say GPU up to 320mm, and the GPU may be 290mm, so then it would fit. Pretty simple.
To make sure your parts are compatible, I suggest using pcpartpicker.com as it will warn you if there are compatibility issues.