Your router does not use much power and yes, you will also need to connect that to one of the UPS plugs to supply constant power.
I would recommend this
APC UPS. I have four and they are excellent and the software that controls shutdown just prior to running out of power is also very good.
While you can start with that, you may need to add the extra
Battery Pack that attaches to the Back UPS PRO BR 1500VA to provide much longer power. I use the set of those two on my largest server.
The batteries last 3-5 years in my experience and when you need new batteries, they are also available to you from the same site and seller.
On the back you will see that you should attach important battery backup devices to the specific plugs and use the surge protection plugs for things that do not need to stay on during a power outage.