twelve25 writes:
> I'm talking about things normal datacenters use. ...
There are far more users of Enterprise storage than just datacenters, and how many datacenters
do you know anyway?
> ... The fastest thing you'll find in a typical datacenter is 40Gb Infiniband, ...
I'm sure plenty are using FibreChannel aswell.
> ... The server you linked to is PC-based, ...
ROFL!!!! I guess you don't know anything about SGIs and shared-memory systems. An SGI UV
is about as far removed from a PC as a cheese biscuit is from continental drift.
> ... a theoretical maximum of 8GB/s (real world is likely 80% of that). So speeds like you mentioned
> above are only capable in aggregate, which you could theoretically do with thunderbolt links.
The UV is a shared-memory single-system-image machine, an application can exploit ALL the
available bandwidth, I/O performance and RAM if that is possible based on how it's coded, etc.
That's why the Onyx2 could sustain 40GB/sec more than ten years ago. The setup wasn't
exactly spartan by modern standards of course (dozens and dozens of FC links), but it works,
and that was only with a 64-CPU system using the original 1st-gen NUMAlink (1.6GB/sec total
per connection) and the old IR2 gfx. The last generation of the hw from that era was the
Onyx3900 which scales to 1024 CPUs single-image with faster connections, much better gfx
(IR4), support for SATA/SAS (Onyx2 never had that), lower memory latency, better node hop
scaling, much higher memory bandwidth, etc., but no data is available on the performance of
the equivalent O3900-based Group Station because SGI stopped publishing the details after
9/11, for obvious reasons (the plentiful detail in the original PDF I referenced was removed,
replaced with a much briefer PDF, also on my site if you want to check).
Read the following carefully and remember this is from more than a decade ago: the Group Station
could load and display a 67GByte 2D sat image in the less than 2 seconds. The max resolution
supported by the sw was more than 100K pixels across (115K IIRC). We're talking performance
levels that are still 2 orders of magnitude beyond a typical PC, but that's what was possible back
then if you had a budget of several million. Btw, I have a 36-CPU Onyx3800 in my garage (pic taken
of me next to the system a few weeks ago):
http://www.sgidepot.co.uk/misc/Ian_who_built_my_computer.jpg
It was used by Sony Pictures Imageworks. One of their engineers told me they need crazy I/O speeds
these days for some of the more compelx renders they have to do. A single frame might require the
processing of more than 500GB of data. It ain't just datacenters who have to hurl big data round now.
The I/O performance possible on a modern UV is far beyond the realms of thunderbolt. AFAIK
thunderbolt isn't yet designed to scale in this kind of manner, or do you know of any product
available *now* that allows a thousand thunderbolt links to operate in parallel?
Hehe, PC-based indeed... I almost spewed my cuppa reading that one, funniest thing I've read
in ages.
😀
Ian.