If your heatsinks are full of cruft, it can cause the parts they are cooling to overheat. This can cause many problems. Most CPU's and GPU's will start to throttle their clocks when they reach a predetermined temperature. Overheating can also cause your PC to crash / reboot / BSOD etc.
So in a word yes it can have an affect on performance and stability.
To keep your PC happy, keep out the dust bunnies.
Just a word of caution, if you are using canned air to dislodge the dust from heatsinks and fan assemblies, don't let the fans spin when you are hitting them with the air. The compressed air can get the fan spinning so fast that it can damage the bearing.