So you have found for sure the noise is from the CPU cooler fan. Further, you also know how to remove that from the heatsink and replace, WITHOUT removing the entire cooler system from the CPU. You best solution by far is simply to replace that fan. It really is NOT possible to disassemble current fans, clean and oil them, and re-assemble.
Most CPU cooler unit fans can be replaced with similar fans from several makers - NOT necessary to get an identical replacement from the maker. Most are in "standard" fan sizes like 80mm, 90 mm, or 120 mm. But you do need one that is rated for higher backpressure to force air through the narrow spaces of the heatsink fins. So you need a "pressure" fan, not an "air flow" fan. And ideally it should be a 4-pin PWM type, not 3-pin UNLESS your old unit is a 3-pin fan. If you need help finding a fan to meet these needs, tell us the maker and exact model of the CPU cooler system you have, or at least what the label on the FAN says. Oh, and the size? The spec is the length of one SIDE of the square fan frame, not the distance between the mounting holes.