You're going to have to be extremely careful.
When you apply Thermal paste, it usually comes in a little tube.
The biggest mistake that people make is using the whole friggen thing. lol
The best usage is only a tiny drop on the CPU lid about the size of a grain of rice. The pressure from applying the heatsink will spread it evenly over the lid itself. But you can do this manually with a plastic card or a piece of cardboard.
Thermal paste is non-conductive, meaning that it won't carry an electric current.
However, leaving it there is a very bad idea, and you want to clean as much as humanly possible without damaging the socket or the pins.
For an Intel CPU, the pins are on the motherboard, and cleaning them can be very difficult. On the bottom of an Intel CPU is a flat surface with little gold contacts, you can simply wipe this down with a non-abrasive cloth and some isopropyl alcohol.
AMD CPU's still have the pins on the deck of the CPU, and are harder to clean.
The only right way to clean off thermal paste is to be careful as hell...
Here is a short video that might help you.
https://www.youtube.com/watch?v=ZfjTjpIDnOg