The only factor would be what the dust consists of.
If it's home or general office use, non industrial, it's fine. Might get warmer than usual inside the socket, but it would have to be plugged.
If it's industrial, manufacturing, then excessive buildup may cause shorting.
If bad enough it could make it inoperable or intermittent.
But it also depends on how the dust got there. If it was laid down after CPU insertion and being secured by the socket it's not likely to do anything since the dust particles are far too large to work its way between the pin and the tightly clamped socket contact to interfere with a connection. It would have to be a horribly dusty socket and/or CPU that were being mated, in which case some dust particles might interfere with a good connection between a pin and contact.
Massive accumulations of dust (especially with cat or animal fur mixed in) on CPU cooling fans and cooler fins will block airflow, making it heat up. Modern CPU's adjust performance based on core temperatures so it will degrade if that sort of condition exists. That's far more likely than getting it on the pins after installation