Yup, going to push ahead with the return...Well, that CPU is 12 years old. That's an incredibly old platform and it's highly likely that there are going to be some modern applications and software that it won't run. Since you are having problems as well I'd definitely return it if they are willing to allow you to. Old is one thing, but that is seriously approaching the "unusable" category.
Any suggestions on what I should look for in the sub-£100 category to give it a bit more longevity?!
It's basically a MAME machine so doesn't need a lot of umph