the router should be the very first thing plugged to your provider.
it has two sidees : a WAN side : pluged to the provider
and a LAN side, wich all your stuff connects to.
simple routers only have a lan side and a wan side
combined routers often also have a switch on the lan side (so you have several lan ports), and also a wifi radio, pluged on the LAN side.
so, the router in your basement is a router + switch + wifi access point, combined.
if you need aditionnal wifi coverage, the best thing to do is to add another access point. configured as a "dumb AP"(i.e. not fulfulling a router's role, but more like a wifi switch)
You can buy such "dumb aps", the bests are the unifi from ubiquiti. Or you can buy an existing router and configure it like so (often it's not possible in default software)
then you connect this AP to a lan port on your router.
simplest thing is to use an UTP cable (also called network cable or RJ45 cable). an electrician or network guy can do this for you. Often if you offer like 50-100$ under the table to the cable guy who installs your internet, he would do it.
most dumbs APs can be powered by their LAN port, with an adapter that add 24 to 48v electricity to the UTP cable. So you can have the power adapter next to your router in the basement, and have your dumb AP neatly installed on the wall or ceiling
but if it's not possible you can reuse existing cables with adapters.
like reuse coax cables as suggested , or electricity cables with "powerline network adapters". but you have the cost of buying and maintenance and additionnal complexity.
Source : it was my former job, plan and install wifi in public places. i did exactly what i explained.
I installed the wifi in the planetarium of Montreal, several libraries, arenas, etc
all with unifi APs and custom openwrt routers
zappublicwifi.ca