There is another factor to consider when it comes to 2.5" vs. 3.5" drives. When a drive fails, the whole thing needs to be replaced. If you are using a RAID 5 array incorporating the failed drive that needs to be rebuilt, then the more capacity the drive has, the longer it will take to rebuild the array. And the more risk there is of data loss (or at least a need to retrieve-from-backup) caused by a second drive failure. Having smaller drives - and more of them - means that the amount of time it takes to recover from a drive failure is less. Yes there will be more frequent failures because of the increased drive count, but each failure won't be so catastrophic in its impact. Imagine a scenario with three 100 TB 3.5" drives in a 3-way RAID 5 array, for a total of 200 TB. If one drive fails, you need to rebuild 200 TB worth of parity data. Now imagine the same data stored on twelve 25 TB 2.5" drives in four 3-way RAID 5 arrays (RAID 50, I think?). If one drive dies, you only have to rebuild 50 TB worth of data instead of 200 TB. The storage ratio of 3.5 to 2.5 is somewhere between 2:1 and 4:1 I think. In terms of physical space, a 3.5" drive consumes roughly 1" x 4" x 7", or 28 cu in. (allowing ~ 1" for connectors) A 2.5" drive is 2.75" x 5" x ~0.5", or ~7 cu in. (again with ~1" for connectors), so it occupies 1/4 the physical space and provides 1/4 the storage or better. You do need more SATA or SAS ports, but in return you get better performance (2-4x the number of drives delivering data), and faster rebuilds / recovery. And yes, recovering 50 TB (deliberate exaggeration to make the point clear; not a typo) will take a long time, but recovering 200 TB will most certainly take a lot longer, slow down access to a lot more data, and jeopardize more data.
I think the only reason that normal desktops haven't moved in this direction is a marketing issue. People have the preconception that 3.5" HDDs are faster, more reliable, and generally speaking, better than 2.5" HDDs. And for single drive comparisons that is generally true. But for specific capacity points, a 3- or 4-drive RAID array performs better and provides drastically more security than a single-drive, and a set of RAID x0 arrays will also perform better and provide more security and availability than a single RAID x array. Server offerings designed for 2.5" drives are already commonplace. It's just the desktop market that lags behind, and that is because we all buy cases with 3.5" drive cages, and buy 3.5" drives with which to fill them. A market leader could start marketing compact desktop cases (such as AIOs require) designed with 2.5" drive cages, and before we knew it the 2.5" form factor would become the market leader.