You can clean it in the socket, and in some cases this is safer as you avoid bending pins or getting residue on the contact pads.
You can use a paper towel and Q-Tips to remove the heavy paste from the surface and edges of the CPU
For finished cleaning, use a low lint or lint free wipe
For both steps above, use a small amount of rubbing alcohol dabbed onto whatever you will be wiping with. 91% alcohol is best, and can be found at most pharmacies
Other specialized products like Arcti-Clean are also good to use and very effective
Your goal is 2 fold:
1) Don't gunk up or spill tons of cleaner and left over thermal paste on your board or CPU socket
2) Leave a clean, lint free surface on the CPU for the new cooler and TIM