It's called thermal paste for a CPU. A small amount of this stuff is placed between the top surface of the CPU case and the flat bottom plate of the heatsink. It increases the rate of heat flow from CPU to heatsink. Too little is not good, but too much (a thick layer) also is not good.
You need to check for instructions on cleaning off any old thermal paste. You'll have to remove the heatsink again and examine its surface and that of the CPU top. They need to be cleaned of old paste - often with a LITTLE bit of a solvent like rubbing alcohol on a tissue - so the surfaces are smooth and free of old residue. Then READ the instructions on how much thermal paste to apply, and how. Often these depend on exactly which CPU you have (because they vary in surface area). Good instructions often are posted on the website of the paste's maker. Apply just the right amount of paste, then re-mount the heatsink on the CPU (sometimes the instructions advise a small twisting motion) and tighten it down again. Just replacing this paste properly should improve heat removal, but there will be a small further improvement over the first couple of weeks after the job is done.
For reference the next time:
Usually you do not need to replace the thermal paste IF you do NOT remove the heatsink from the CPU. So next time, just unscrew the fan from the heatsink and clean, then replace it.