During boot you should see an option like F2- setup or something of the sort. Hit the button before it boots into windows and you should get into the bios. Find your LPT/COM port setting and set it to "ENABLED". Use the option to save settings and reboot.
Once in windows, if it doesn't detect the ports automatically, go into the device manager and use the option to "scan for new hardware".
The yellow exclamation marks and question mark means the hardware was detected, but the incorrect or no driver is installed for them. If you built this computer, be sure to install the chipset drivers that came with the motherboard. This should fix most of those.