If a semiconductor is run within it's specs, it can essentially last forever... or they could back before everything switched over to tin solder.
But the problem with used parts is that you never know how they were treated before you got them. If you think the seller might have been pushing a chip past it's limits or overclocking, then I would avoid that seller. Also, ensure that the seller knows how to properly ship and protect their chips from ESD.
What -won't- last forever, is thermal paste and electrolytic capacitors. For this reason I wouldn't buy a GPU that is more than a couple years old (even new) unless it uses higher end capacitors and you are prepared to replace the thermal paste... which is usually trickier than on a CPU. That said, GPUs are crazy expensive right now so in this case I think it could be worth getting an older one with the intention of replacing it when prices go down.
Don't buy a used Power Supply, ever. Don't even buy a "cheap" power supply or recycle a power supply from an old PC. Power supplies have big capacitors that wear out and cheaper ones are often advertised way beyond their actual capacity. The risk to the rest of your PC isn't worth saving $50. I would also be cautious of buying a used hard drive as hard drives can fail after only a couple years of use.
You should have no serious problems reusing an old case though. Very-old cases usually were a little harder to install components into, but anything somewhat modern should be ok. With a used case you might not get USB 3 on the front panel, and the case fans may have worn out or gotten noisy, case fans are not very expensive, but factor in an extra few dollars when comparing value.