You need drivers for your USB controller not to mention devices connected to it. You don't need drivers for your speakers because it's a one-way, analog signal being constantly outputed. No communication is neccessary, analog audio signals are all the same.
As for drivers, they're required because the OS itself does not know how to communicate with a device. Think of them as OS-plugins, so yes, they'd be different for IA-64 vs x86-64 since just like any other program, they run code that's either x86-64 or IA-64 (the binaries are different).
As for how they communicate with the device. If it's the same device, you wouldn't need to change your C code assuming everything else in the system is the same. A simple recompile should do.
"We are Microsoft, resistance is futile." - Bill Gates, 2015.