Lets pretend you don't have a router. You plug both ps4 directly into the modem and you have paid the ISP for 2 addresses. Each ps4 now has its own real public routable IP address and will work fine. Be aware these IP are actual internet IP not the private 192.168.x.x that your router give out.
If you use 2 routers or plug the 2 ps4 in directly is not the issue. Putting a router infront of each ps4 just makes things more complex but the end result is the same....each ps4 gets to use its own public ip address. The problem is the ps4 assumes it has full control over the ip address and if you only have 1 actual ip address the ps4 will fight over it since they both assume they are the only ps4 on the network.