In a nutshell: Optimization.
On a console front (XB1 or otherwise), developers only have to code for a very limited number of hardware configurations, on hardware specifically designed to run games (2.3GHz 8core chips etc) - at set quality presets/resolutions; the couple of XB1 variants, PS4 etc. <10 different, unique(ish) configurations..
Compared to the PC equivalent of thousands (or more?) different configurations, it's just easier (not "easy", just easIER)
Server chips are a horrible comparison as, outside of core count and clock speed, there are barely any comparables between them; they're designed for entirely different workloads.