Thanks all for the help.
After I installed Speedfan, I sat the fan speed to 65 percent. Now the fan speed ranges from 1900 to 2400 rpm according to CPU load and ambient room temperature, just like the old days. It seems that the software that is controlling CPU fan speed is multiplying the speed for no good reason.
As for the heatsink and the thermal paste there is nothing wrong with them since the temps are very good now which means that thermal conduction is adequate: 47 idle at 2100 rpm. As for a liquid system there is definitely no need for it regarding this particular problem.
For now the problem is solved, but I will keep Speedfan running for a week and see what happens after that. Maybe I will have to run it permanently, or maybe the fan will break eventually!
As for the BIOS, there is an option for something called Qfan. Anyway, enabled or disabled the problem is still there.
My system specs:
AMD Athlon X2 4800+ at 2.4 GHz, 2 x 1GB DDR-1 400MHz RAM, 650 watts Thermaltake PSU, Radeon 4890 1GB, ASUS A8N32-SLI DELUXE mainboard with nVidia nForce4 chipset.