Physically disconnect the device if you cannot uninstall the driver. The ease of that depends on what desktop or laptop you have. Most laptops allow you to turn off the wireless instead of having to do a physical disconnect. With any luck you can then remove the driver.
If you have any further questions, provide more information -- like computer brand and model, adapter model if known, and OS version.