Even if it's connected, a lot of contractors mistakenly wired Ethernet as if they were phone lines. With phone lines, you can wire all the ports in the house together using the same wire. You cannot do that with Ethernet - each cable can only connect two ports.
So you'll have to check for that possibility too. How many ports are in your house, and how many in the network closet? If there's one port in the network closet but multiple ports in the house, then it was wired up wrong. When correctly wired, if you have 10 ports throughout the house, then there should be 10 ports in the wiring closet.
It's also possible that your basement is not the network closet. There may be another location you haven't found that's the network closet. You would need to install a switch there, and plug cables from the switch to every port coming into the network closet. Then when you plug the router in the basement into the wall, it gets a connection to the switch, and the switch distributes that connection to all the other ports in the house.