Wireless technology is also vulnerable to trouble from a single device that connects slowly. The entire wireless connection will only go as fast as the slowest device connected. So, if one device is having a hard time connecting, for example if a TV is between it and the AP, it will slow everything else down. It only affects one channel at a time, so if you're Xbox is connected to the 5GHz band while the rest are connected to the 2.4GHz band, the Xbox wouldn't be affected.
So, to troubleshoot, try these steps:
1. power down all wireless devices and reconnect them one at a time, starting with the devices closest to the AP, then test the speed after each one powers up.
2. if it is slow right away with only one device you know isn't...