I am not sure about your particular ISP but most ISP's give out at least 2 IP addresses.
In that case wouldn't it make more sense to put a small switch directly after the modem and then connect each of the routers to a port on that so that they can both maintain their direct access and be...