No. Drivers only affect cards after POST, not before. You should be able to boot into bios even without specific drivers installed, bios is vga and native. Cpus without detected igpu won't be given the option in bios, only APU's will, as to which is primary and which is secondary video output.
If you aren't clearing cmos before swapping, you may be getting a conflict in bios as that video setting is more than likely set to use the igpu from the APU primary. Cmos is a list of bios instructions, so on boot the cmos comes first and skips much of bios functionality. Clearing cmos after cpu swap will force bios to start from scratch and physically discover everything new. Reset does the same function, windows shutdown does not, it saves cmos list for faster boot times.