Xeons are processors made for Servers and Workstations. They support larger amounts of memory, ECC (memory error-checking), multi-CPU configurations. They are also certified for 24/7 workloads.
There are some drawbacks for consumer use, though. Since they are reliability oriented, most Xeons will simply not work out of spec (meaning they will not overclock or use XMP profiles). Some E3-1XXX Xeons are very similar to their desktop counterparts and actually come unlocked, but this is more like an oversight by Intel than a feature.
Your assumption is right, being a server/worstation product, they are usually a worse alternative for gaming, being more expensive. There are some very specific models that can be actually viable for consumer usage, with a lower price point.
My main build runs a Xeon E5-1660 v3, which is essentially a server version of the i7-5960x. The reason I got it was that it was a pull-out from a server at work. It does not overclock and won't boot if I change a single MHz in memory spec. Left at stock, I usually leave it running, with a week or so of uptime before I reboot it for updates. It runs Windows Server 2016.
I do gaming on this PC too, which works flawlessly.
The reason some professionals prefer Xeon workstations is reliability. ECC may prevent the occasional flipped bit that can corrupt memory (look up "
cosmic rays". No, I'm not kidding). Depending on your line of work, a single hard reset may justify the cost. You can also amass an ungodly amount of RAM with Xeons (mine can do 768GB). This can be necessary in certain workloads.