I typically use a mixture of alcohol wipes and pressurized cans of air, although isopropyl alcohol also works.
That is called thermal paste. You can remove it with an alcohol wipe if you want and apply a new coating of it, but you'll need to pick up a tube online, although they are typically quite cheap anyway, about $5 for the good brands like the Arctic Silver 5.
You only need to put on a pea sized dab in the center, and attach the cooler as normal.
The mixture will flatten and spread across your CPU's integrated heat spreader.