Don't buy a used Power Supply, ever. Don't even buy a "cheap" power supply or recycle a power supply from an old PC. Power supplies have big capacitors that wear out and cheaper ones are often advertised way beyond their actual capacity. The risk to the rest of your PC isn't worth saving $50. I would also be cautious of buying a used hard drive as hard drives can fail after only a couple years of use.
You should have no serious problems reusing an old case though. Very-old cases usually were a little harder to install components into, but anything somewhat modern should be ok. With a used case you might not get USB 3 on the front panel, and the case fans may have worn out or gotten noisy, case fans are not very expensive, but factor in an extra few dollars when comparing value.