I don't think it's necessary to remove the driver. Just run the installation of the new one and you should be fine.
I've done that plenty of times, and the only issues I had was that some stuff required repeated attempts for them to be installed.
The last time I tried removing the drivers, it bricked the laptop.