How do I hook up my laptop running windows 8.1 to three Raspberry pis using just a router? I want to be able to access any of

The work "access" has many meanings:
- your RPi's have web server you want to access
- your RPi's are sharing files over SMB
- your RPi's are running application which needs remote control (eg XBMC/Kodi)

ANy way - if you have router already, and your PC and RPi's are connected to that router, they already have their IP addresses allocated, you just have to discover them. Otherwise - follow @Kanewolf, assign each RPis and your PC static IPs from same network, then use browser/file explorer/application/ssh/telnet to access them.