What I recommend is buying used, especially for older folk who care very little for the age or look of their PCs as long as they work. Look for old workstations like HP's Z series and Dell's Precision line:
HP Z400 - $165 - Xeon W3565, 4GB DDR3 RAM, 1TB HDD, Windows 7 Pro, Some GPU. Add a cheap SSD and this old horse will gallop as fast as PCs far above it's price. You can add some RAM later on if need be.
Dell Precision T3500 - $195 - Xeon W3550, 10GB DDR3, 1TB HDD, Quadro 2000 GPU. Add Ubuntu and you're all set. An SSD is optional, but will give you a speed boost.
Copy the link address, as Tom's editor does not like when multiple ebay links are posted.