The adaptor doesn't need any drivers, the games need drivers to use wheels, since you get any input through drivers are ok.
It's either the adaptor or the wheel itself, probably some pin on a plug/cable is loose and not staying connected al the time.