Nvidia and AMD Inside Titan: Fastest Supercomputer on Earth

Page 2 - Seeking answers? Join the Tom's Hardware community: where nearly two million members share solutions and discuss the latest tech.
Status
Not open for further replies.
[citation][nom]blazorthon[/nom]I'm pretty sure that "excelerative" isn't a word and even if it was, it'd probably be spelled more like accelarative. ...[/citation]

Yeah, I made it up, and I was lacking Earl Grey in my system - your spelling is more logical,
so +1 for that. :)

Ian.

 
[citation][nom]ohim[/nom]what , you have a supercomputer in your bedroom and want that or what ?[/citation]

I have one in my garage, but it's a bit old (SGI Onyx3800 IR4, 36-CPU, 72GB RAM), and until the
power company rewires the place with 50A 240V, it's staying off for the time being. 😀

Ian.

 
[citation][nom]whiteodian[/nom]You already can. Go to the top of the comments and choose "Read the comments on the forums" Then go to your comment and click the quickedit button.[/citation]

Oh!! I had no idea the facility was available, so thanks and +1 for that! 8) I removed my typo post.

Ian.

 
[citation][nom]RealiBrad[/nom]Yes, imagine the amount of porn you could watch with a super computer....[/citation]

Except by the time the system was fully booted, you'd be totally bored and fallen asleep... 😀

Large systems with lots of ECC RAM take aaaages to pass all the stringest startup RAM tests,
unless they've been bypassed (which isn't a good idea).

Ian.

 
OMG now it makes sense this is how Intel is gonna be brought down to earth.... AMD & Nvidia are gonna have to work together to kill Intel!!! Yes Yes Yes whaaaaaa
 
[citation][nom]ohim[/nom]Am i the only one surprised that the worlds`s fastest super computer has AMD CPUs at heart ?[/citation]
The only reason it had AMD's were that the Intel counterparts were $1000+ a piece instead of ~$250. When you have 50,000 of these cpus stacked in a supercomputer it adds up very quick.
 
I'm wondering if you said that because it's just a regular thing that happens (supercomputers beating out each other) or because you've heard news of a better one in the making. :)
 
[citation][nom]the_brute[/nom]@IanNot many people get a chance to ever play with a super computer, and have NEVER had to wait for these beasts to start.[/citation]

Good point. 😀

Bargains to be had though, one just has to keep an eye out. I know someone who won a 24-CPU Onyx IR rack on
eBay for the lunatic sum of just 50 UKP. My Onyx3800 cost more to ship from CA to the UK than the amount I paid
for the system. I think the NASA auction site and others often list this sort of thing (more common in the US), but
they don't turn up on eBay so often now. My first large systems were a 24-CPU Onyx RE2 rack and a 24-CPU
POWER Challenge rack, each of which only cost me 200 UKP (they were about a million each when new; 2GB RAM
was huge in 1993); they were used for car crash simulation by Ford and BMW, then for crunching seismic datasets by
a geo-sciences company, before I bought them in 2004 (these are very typical uses of supercomputers). I sold the
Challenge, just have the Onyx now. They are of course no match for modern hardware (my 2700K is faster than the
entire Onyx rack), but fun to mess around with.

Getting the Onyx3800 delivered was tricky though, sometimes this stuff requires a modicum of dedication, and a
friend who can be suckered into helping out... :}

http://forums.nekochan.net/viewtopic.php?f=3&t=16720350

Mind you, working on old large systems can lead to uber frustration when the slightest issue can drive one nuts trying
to trace the cause, at which point the following idea of how to deal with the machine is all too likely...

http://www.sgidepot.co.uk/misc/ian7.jpg

😀


But sure, when it comes to obtaining old supercomputers, seek and ye shall find. Just contact local industrial companies,
defense places, universities, etc. I focus on SGI stuff, but I'm sure all the various brands/models are out there to be had
if one looks for them (IBM, SUN, HP, etc.) Just don't expect your gf/wife to smile at you when a 1600lb crate gets
delivered. 😀

Cray is a bit odd though. They used to have a thing of always wanting their hardware back if it was no longer being used.
Don't know if they still do that.

Ian.

 
@Ian
I see... You collect stuff, let it earn up more value with time, then sell it (or just sell it to anybody who'd buy it for the right price). Not bad, plus the pleasure of having a more exotic variety of hardware pass through your hands. 😉
 
[citation][nom]captainblacko[/nom]Are these off the shelf GPU's or some sort of mad scientist / Frankenstein version of say a GTX680?[/citation]
Titan uses the Tesla K20X which is powered by gk110. The GTX680 uses gk104, which is a smaller graphics/gaming oriented derivative of the Kepler architecture. The Tesla K20X has 2688 CUDA cores enabled running at 732 MHz, which is connected to 6GB of GDDR5 memory through a 384 bit interface, clocked at 5.2 GHz. Its theoretical DP performance 1.31 TFLOPS and its SP is 3.95 TFLOPS.
 
Really, Tom's? "petaflops per second"? FLoating point Operations Per Second, per second?

You guys really don't need to dumb it down; it just makes you the dummy, when you do.
 


Actually the big-iron SGI stuff from the Origin3000 era has devalued massively since I bought the Onyx3800. I should have tried
to sell some of it straight away, but never got round to sorting it out, always something else needed doing. These days the RAM
kits are more valuable than the CPU bricks. 😀

I expect it's the same with systems from other vendors like SUN. Medium-age hw is not that valued, but once it becomes old enough
and parts are hard to find, item values rise rapidly, assuming the demand is still there. A typical CPU brick for Origin3000 isn't worth
more than a few hundred $ at most now (low-spec bricks are worth nothing, nobody wants them), whereas certain much slower CPU
boards for the older Onyx/Challenge series are worth $3K+.

You might find it interesting how these systems work, gives an insight into the early days of parallel systems, shared memory, the
high-end gfx which eventually led to the GF256, etc. See:

http://www.sgidepot.co.uk/sgi.html#ARCHTECH
http://www.sgidepot.co.uk/sgi.html#TECHREPS

eg. check out the Onyx2 Technical Report, Infinite Reality Technical Report and RealityEngine in Visual Simulation (Technical Overview),
and the various Origin documents. i wrote a specific page for the newer O3K series:

http://www.sgidepot.co.uk/origin3k.html

Perhaps one of the most interesting documents on the Origin2000 series is this one:

http://www.sgidepot.co.uk/onyx2/groupstation.pdf

Note the performance the system is capable of. After 9/11 the document was rewritten & greatly simplified, ie. the performance of the
later 3000-series systems was never disclosed (in other words, defense imaging using Onyx3900 and IR4 gfx).


Anyway, SGI bought Cray at one point, so for a time there was a back & forth of ideas, the most persistent being the CrayLink (now
called NUMAlink) which is still used today in the UV series (see sgi.com/uv).

Ian.

 
@Ian
Interesting... Thanks for sharing those links. :)

Maybe you should hold on to those systems longer. It is doubtful that someone would buy them for performance, but more for just collector's value. If ever, you might find a museum in the future that would pay top-Dollar for them. Or maybe even your children may (if you have any). I would recommend putting those systems away in one of those storage places that have cargo bay doors, but I don't know how much they cost and if you watch shows like "Auction Hunters," you'd see what happens if you forget to pay the rent (at least I think that's what happens in those shows). :lol:
 
army_ant7 writes:
> Interesting... Thanks for sharing those links. :)

Most welcome!!


> Maybe you should hold on to those systems longer. ...

Oh I intend to. I'm never going to sell the Onyx rack, one of the desksides, Crimson or Onyx2, those are for permanent keeps. 8)

I'll sell part of the Onyx3800 in time when I can, probably some of the CPU bricks, power bricks, PCI bricks, etc. (I don't need
48 PCI slots! Hehe). But I'll keep at least a few, enough to run maybe a 16-CPU system so I can do Inferno benchmarks, etc.
The system came with a full suite of HD hardware for the gfx module aswell (the system belonged to Sony Pictures Imageworks),
so I'll probably keep all that too, and the dual-pipe gfx unit can drive 8 HD displays at once. 😀 Actually btw, some companies
do still use old top-spec Onyx3K systems with Inferno for film work because despite the old-level CPU power (Inferno was
deliberately limited by Discreet to using no more than 8 CPUs), they're still quite good for compositing when fitted with IR4 gfx
(1GB texture RAM and 10GB video RAM per pipe, up to 16 pipes per system), but most places that never had IR4 have long
since sensibly switched to HP XEON systems with Quadro, etc. (the old IR3 gfx is not as useful, only 256MB TRAM and 800MB
VRAM per pipe).


> It is doubtful that someone would buy them for performance, but more for just collector's value. ...

Indeed, though there is still commercial demand for some parts of this type of system, usually power supplies, RAM, cooling
fans, and especially the raster manager boards for IR4 gfx (those are worth $3K+ each). I expect this is true of any vendor's old
supercomputer products, certain parts continue to be required for maintenance purposes. Many companies are still using SGIs
from more than 10 years ago, especially in simulator/training roles.

Ideally I'd like to upgrade the system to Onyx3900 (a single CPU brick contains 16 CPUs, max 1024 CPUs in a single system,
though NASA had one with 2048), but they are far too expensive even today.


> If ever, you might find a museum in the future that would pay top-Dollar for them. ...

I did try contacting such a museum here recently about something else, but they never replied. Perhaps some day.


> ... I would recommend putting those systems away in one of those storage places that have cargo bay doors, ...

My garage suffices atm. 😀

Such places do exist, I looked into them the last time I moved house, but they're expensive.

I have lots of other SGIs and parts, plus some other mid-range systems (16-CPU Origin300, quad-CPU Onyx2 deskside, etc.) See:

http://www.sgidepot.co.uk/misc/garage11.jpg
http://www.sgidepot.co.uk/misc/garageNov2011f.jpg

Pity my neighbour needs his garage... :}


If this sort of thing interests you, ask around industrial companies, universities, defense places, etc. Often they're glad to get rid of old
stuff as it just takes up space. Apart from SGIs, there are Alpha systems, HP, SUN, etc. SGIs have always been more interesting to me
though because of all the gfx/imaging/video things they can do aswell as number crunching. Quite a laugh to run Maya on a 36-CPU
machine. 😀 And from what I've been told of US companies, they're somewhat more willing to accommodate hobbyists looking to obtain
such things. In Europe, all too often this sort of hardware is simply thrown away. Don't be shy about contacting these institutions, just
phone them up, ask to speak to an admin or technician, etc.

At one point I even had a UK govt dept. ask me if I was interested in a huge 144-CPU SGI, but that was way too much, I didn't even
have a garage space at the time.

Ian.

PS. One site definitely worth watching is DoveBid: http://www.go-dove.com/en/

 


Heh, best super computer titles often don't last even a single year. Faster super computers are always coming out.
 
Status
Not open for further replies.