Pc's that are plugged in are Not Off, unless physically turned off at the psu, or unplugged.
Depending on which Windows On Timers (in power plan) are active, the motherboard, USB, and sometimes mouse, keyboard and even Wi-Fi/ethernet can and are usually still powered. The pc is technically still On as long as the psu can pass power through. Only the OS is technically Off.
So your USB dongle will always have power, unless the psu is fully Off and not in Standby mode (Windows Off). Just as you can plug a cell phone into a USB when the pc is 'off' and still charge the battery. You can turn that option off in Windows power plan and/or bios by changing USB ports to Selective or Off (not recommended).
USB and hdmi and DP can require a 'handshake' to activate the port. This always happens when a USB item is first plugged in, windows goes through a recognition process and allows the USB to function. If you power 'off' the USB in bios, Windows may or may not recognise the USB as present when you push the power button, requiring you to unplug and replug the usb item. That gets tedious, especially with a Wi-Fi dongle. Best to just let it be, it's not harming anything since it's operating according to design.