ARM chips could have a great co-existence with the current data center hardware with 3 markets:
1. IPMI processors - this could be the first break-in point. Currently, server manufacturers use some pretty bad hardware for this purpose. With ARM chips, it could be done with an industry standard, less power, and easier maintenance.
2. hardware acceleration for RAID arrays - current chips used in RAID arrays have less calculation speed and use far more power than ARM chips. this could be the second break-in point
3. IP KVM management - for those systems without IPMI, management can be trouble, and many don't even like KVMs (like IBM P series machines and many HPUX machines) Having a system with constant responses for the keyboard and mouse being connected, so the system doesn't drop keyboard and mouse support, would be vital for constant support. An ARM based machine could maintain this and allow for remote IP KVM management that could be handled from any other computer. It would save a whole lot of frustration for admins.
Sure, doing mainstream server duties would be nice for an ARM based system, but that will take a while to break in. Using these markets to break in first would be a smarter decision. Then, once the company is profitable, start pushing for high duties. This is how Intel got into the server industry, starting way back with 286 machines being used as local access terminals.