Yes just use any router and connect it to the box on the wall. Unlikely but you may have to tell the ISP you changed the router, then again if you are returning their router they will know.
DO NOT just assume that the ISP will allow you to change the router. Some do, but only with
prior notification/request from you, some absolutely will not.... mostly because they will want to continue charging you the monthly rental fee for it
just retuning their router will
NOT usually be taken as an automatic indication to them that you are replacing it, most will just assume you are cancelling your service, which would be a major PIA for you......
Call them before you do anything, and write down the name & number of the person you speak with, just in case they conveniently "
forget" about giving you permission to make the changes !