If it's happening in Safe Mode too, it's pretty safe to assume it's hardware related...
I'd normally suggest you breadboard (ie back to basics, just the CPU + HSF, Mobo, 1xRAM and PSU connectors) but given the lack of onboard graphics with that chip, there's not too much you can remove. Replacing the HDD is a good start, and the most likely culprit in this situation.
If you're not prepared to wait for your new HDD and want to keep trying options, I'd suggest you remove all but 1 RAM module and see if the issue continues.
Technically, any of your components could be causing the freezes, but if the temps are normal on the CPU & GPU, it's less likely to be them.
Another option could be the PSU. I've never heard of the XL model? It's possible there's inconsistent power, or power spikes/surges, although surges would likely manifest itself with a dead PSU, not just freezing.