First, I think you've gotten lucky enough that the wiring was done correctly. Most of the time this is the first problem.
That grey wire looks like it is coming from a dsl modem to an ethernet jack in your box that on the other side is connected to your router.
To get wired connections to all the other rooms, you need to move your router to this box and connect the gray wire to the router directly and then connect from the router to whichever ports you want to have Internet. If you run out of ports on your router, you will also need a switch.
But as bill001g pointed out, wifi won't be good with your router in the box if that's where the wifi is. This is why it's useful for routers and wifi access points to be separate devices--because once you have Internet in the house all the jacks, you can put access points wherever you want them.