All kernel-mode drivers must be 64 bit and compiled against the driver API of your particular Windows major kernel version. Certain elements of user-mode drivers may be 32 bit but are usually 64 bit as well.
Application wise, 64 bit windows has full compatibility with 32 bit windows applications and APIs. This means that there is no harm in installing something like the x86 Visual C++ runtime components as many of these may already be installed but outdated.
It is recommended that 64 bit applications be installed where available but when not available a 32 bit version will work fine.
It is important to note the distinction between kernel mode drivers, user mode drivers, and applications.