Could be just a matter of the reflectivity from the desktop to the receptor on the mouse.
Desktop reflects enough light for the sensor to respond. Not so on the mouse pad(s). Should not be that way - something is not right.
It could be broken but perhaps just an intermittent connection that comes and goes. I would return the mouse.