You can connect though a switch but you won't be able to have both on the internet as the modem will only with with one computer at a time. If you don't want to use a router you can setup Internet Connection Sharing on one system and connect the second one though that.
Setup a static IP on the computers 192.168.1.2 for one, .3 for the other one.