Most graphics cards have a hierarchy of which outputs they send out to first. This can usually be changed and adjusted in the driver settings of the amd or nvidia card. Your windows main monitor( the monitor you told Windows to display as primary) and the monitor input in the graphics card primary slot (the physical input on the graphics card that your monitor is plugged into) are different. If possible, switch the cables around from the two monitors and you should be good to go.
EDIT: Unless your second monitor is plugged in with VGA, you shouldn't have any performance issues switching them over.