The mouse is the transmitter while the usb dongle is the receiver (a mouse doesn't receive anything beside some configuration settings, it sends a signal to the computer) and the headset is a receiver. So unless your mouse is powerful enough to create interferences between your headset and its transmiter (which is extremely unlikely), it's not the source of your problem.
However, the main source of interferences in a house on the 2.4GHz bandwidth is the WiFi router. So make sure your router is far away from your headset transmiter, or disable the 2.4 GHz on your router if you're not using it (most people use the 5.1 and 5.2 GHz nowadays). I had a similar problem with my Sennheiser WiFi headset and I fixed it by disabling the 2.4 GHz on the router that was right next to the transmitter tower. But you can just move it away if you can't or don't want to disable the 2.4.