Home-Built Supercomputer for Scientific Computing

[tsnor]

like others earlier I strongly recommend looking at CUDA. CUDA uses

Start here: http://www.nvidia.com/object/computational_fluid_dynamics.html

Then check your software to see what is supported.

If you care about your results, consider waiting for the next gen from nVidia codenamed “Fermi”. It actually has error checking so you won't need to re-run your code a few times to see if you got the right results or if a sunspot flipped a bit on you. (RAS of current accelerators is mostly missing because they are built from graphics cards that don't care about data integrity)

Suggest: Append to CUDA forums describing your application needs in CFD. Ask for config help, verify software works as advertised, check benchmarks.

Suggest: Start with a 'gamer' PC of your choice. Specify a high end nVidia graphic card. Cost will be $1K to $2K total from Dell or some specialty house. Get the software you need working with CUDA. If the performance is not there then you can consider higher end solutions. Don't throw $20K at this problem until you understand the ups and downs of the $1-2K system.

 

[LEANGREENCOMPUTINGPOWER]

i have some dual nehalem computers listed at ebay... Engineering Samples

remember MATLAB's software is CUDA and OPENCL ready

I have 6Gbit SAS on a board coming in with ISTABUL twin from SUPERMICRO

or TYAN 8-way 84XX series chassis's ready to ship also

omces.com

call mark at 3148450695 for any specs

 

[LEANGREENCOMPUTINGPOWER]

REMEMBER I can build a 10TFLOP machine with the new supermicro board. Screamers here...

LEAN GREEN COMPUTING POWER... mark

 

[chanceusc]

Well right now, I am a graduate aerospace engineering student at Arizona State. Everything I have done thus far has been on my personal laptop ( not a high performance machine at all...) and on standard desktop computers. Although ASU does have a very good HPC facility, I haven't had the privilege to use it yet ( I think the Saguaro super computer is number 150 or so in the world).

Any way, Im not immediately in the market for this hardware as i wont be graduating for another year or two. But, Im starting to look into formulating a business plan for a CFD outsourcing business. For this type of thing, computing power is a major consideration, and Im trying to figure out if it is feasible for me to acquire the necessary hardware at a decent price. Ill admit, Im still new to this type of computing and there are still large gaps in my knowledge. What I do know is, I do enjoy the coding involved with CFD and I enjoy the results even more! That being said, I think there is a market for CFD simulations.

Sorry, kind of got of topic there, but you should know what my intentions are and a little bit about my background.

answering your questions...

Machine Currently being used:

Gateway tablet PC, win xp, 1.5 Gig Ram, 1.74GHz intel mobile cpu
like I said, not a great machine, but does the job for very simple code. I run MATLAB most the time and it is a very slow running language from my experience. It would be completely inadequate for anything really meaningful.

REAL*8 or REAL*16:
No clue! but ill look into it. Im not a fortran guru (yet) but If its as simple as hard coding REAL*8 , no problem right?

Colleagues Running i7?:
Right now my colleagues are fellow students, so no, they are running the same type of garbage I am.

No need to put me in contact with Marketing Manager as Im not immediately in the market but thanks for the offer!

Stupid Question for you. So for this type of system, would it consist of multiple i7's or just 1? I know they are multi core processors but am confused if more than one processor is actually used.

I very much appreciate all of the help and insight you are providing. You are helping me learn a lot.

I havent bothered to read all of the replies to this message so its quite likely they have been addressed already. However, I would strongly suggest that you look into cloud computing with Amazon EC2 http://aws.amazon.com/ec2/ or Microsoft Azure http://www.microsoft.com/windowsazure/products/. Cloud computing would likely save you a great deal of money as you could run the applications as needed (assuming you are conducting simulations or some other quantitative calculations). There will be an initial learning curve, but imo, it would be worth it.

 

[ragvam]

for a real personal super computer under 10k USD try nvidia tesla super computer its really amazing

Get your own supercomputer. Experience cluster level computing performance-up to 250 times faster than standard PCs and workstations- right on your desk. The NVIDIA® Tesla™ Personal Supercomputer is powered by up to 960 parallel processing cores and 16 GB of dedicated compute memory to solve large datasets. Based on the revolutionary NVIDIA® CUDA™ parallel computing architecture, the Tesla Personal Supercomputer puts nearly 4 teraflops of computing muscle at your fingertips.
Up to 960 Core Parallel Supercomputer on Your Desk, 250 Times Faster than a PC.
Up to 4 teraflops of compute capability for under $10,000.
Computation comparable to a large server cluster that fills a room.
250 times faster than a multi-CPU core PC or workstation using traditional serial computing.
GPU parallelism designed to execute tens of thousands of concurrent threads.
Up to16 GB of dedicated compute memory meets the computing requirements of large datasets.
Single- and double-precision floating point.
Scalable architecture meets computational demands of complex scientific applications.
Based on Revolutionary NVIDIA® CUDA® Parallel Computing Architecture.
Unlocks the power of GPU parallel computing.
Over 25,000 application developers worldwide using CUDA.
Over 100,000,000 CUDA-enabled processors shipped.
Accessible to Everyone
Supercomputing available to anyone, anytime.
Iterate faster; discover more.
Computing whenever you need it. No need to ask permission to an administrator.
Buy it anywhere, put it anywhere.
Plugs into standard power strip.

http://www.connoiseur.com/NVIDIA_Tesla_Personal_Supercomputer-list.aspx

 

[jackalak]

I'm a graduate student at caltech, I know about cuda and gpu's and parallel computing. I do CFD research. The type of computer you need depends on the type of flow you are simulating. If you are simulating incompressible flow, then I would say learn about gpu's and write your own code from scratch and use a FFT solver library that runs on a GPU. If you are modeling compressible flow (hyperbolic equations), then I would say you need to buy a parallel computer made up of i7 chips. If you don't want to build it from scractch, buy a Cray personal super computer, $25000 to $64000. This will allow you to use up to 64 cores I think. If you want to start smaller and just have a desktop. Use an Intel i7 configuration. You need double precision to do compressible flow. GPU's are slower for double prescision...they are getting better though, there also exists no comericial compressible flow software that runs on GPUs, you'll have to make your own. Wait a couple years and check up on that though. However, also buy that time there will probably be 32 core processors on the market, and those might be better for CFD than GPUs. Basically right now, for programs based on the MPI model, getting access to a cluster, (64 processors or more) is ideal. I'm assuming Fluent runs on clusters here. Would you hardware experts out there agree with this? I'm also interested in your opinions.

 

[jc5706]

My honest and sincere advice is that if you are a recent graduate and want a computer to do CFD at home, just buy a good PC, don't worry about clusters, GPUs and servers.... I suppose you are not intending to model the airflow in a commercial plane with a detailed geometry and under project time pressures, right? Everybody waits for CFD results.

 

[tommyk19]

I would start with a careful inventory of all production software
you intend to run on a regular basis.

Let me give you a good example from our daily workload:
after we installed COPERNIC desktop search software
on a quad-core Q6600 workstation, we compared it
head-to-head with a dual-core D 945 in a similar workstation.

The Q6600 finishes just about twice as fast as the D 945
updating the very same 5GB database.

That can only be the result of parallel programming in COPERNIC.

If your software is not coded to exploit multiple cores,
you're wasting your time bulking up on multi-socket systems.
Those are intended chiefly for busy servers with lots of
multi-tasking to process.

After you've completed your analysis of your production software,
the real choice for you is between a single-socket Core i7 machine,
or a twin-socket Xeon with Core i7 architecture.

A single-socket Core i7 machine has 4 CPU cores with hyperthreading:
each socket is THUS capable of running 8 threads simultaneously.

That should be enough for any software you want to throw at it,
particularly if you are only running one copy of your fluid dynamics model
at any given point in time.

Next decision is the clock speed of the Core i7: go for the fastest
because you've got plenty of budget for it, and it was just reported
on the Internet that all Intel Core i7 CPUs are "unlocked" --
meaning they can all be overclocked.

So, buy a motherboard that makes overclocking easy, e.g. ASUS P6T.

As for RAM, X58 chipsets now support either 12 or 24GB of DDR3 RAM:
look into Corsair's high-end Dominator series, with the memory module cooler.
Kingston make engineering samples of 4GB DIMMs x 6 = 24GB total e.g.:

http://www.hexus.net/content/item.php?item=17187


The Corsair power supplies w/ 850 Watts and up should be enough:
they are very highly regarded for high-end workstations like the
one you want to build:

http://www.newegg.com/Product/Product.aspx?Item=N82E16817139009&Tpk=N82E16817139009

I'll leave your graphics hardware choices up to your own good research.

If you are really serious about writing very large data files,
then be sure to give serious consideration to motherboards
and/or RAID controllers that can pump data quickly to and from
SAS (Serial Attached SCSI) hard drives: these come in both
10,000 and 15,000 rpm e.g. Seagate.

Despite what lots of amateurs claim, in ignorance,
a RAID 0 with multiple (4 x or 8 x) HDDs can really move a lot of raw data
very fast: 250-300MB/second is quite easy, and 500MB/second
is within reach, if you know what to buy and how to configure
your storage subsystem. The key is choosing a RAID controller
that does parity computations in hardware, e.g. Areca, 3Ware
or Highpoint's "enterprise" class controllers (there are others).

Out on the "bleeding" edge you will find Fusion-io's ioDrive Duo
(Google that one), or OCZ's new Z-drive (actually just a
Highpoint RocketRAID with 4 x OCZ MLC SSDs inside the
plastic shell).

Lastly, we've had enormous success with ramdisks
by installing RamDisk Plus from SuperSpeed, LLC
in Sudbury, Massachusetts: www.superspeed.com

A little bit of intelligent memory management, tailored
to your application software, can pay enormous dividends:
Core i7 memory bandwidth BEGINS at 25,000 MB/second
(25GB/second), and goes up from there when triple-channel
DDR3 is overclocked.


This is THE hottest computer hardware available
at the present time for workstation-class machines.
AMD still can't even come close.


hope this helps


MRFS

Hi there,

I have found this thread and really liked you reply. Question is if I ask same question now (mean 2012) what would be reply or what would you change and what would you keep on above mentioned setup?

I'm in similar situation so wondering where to start in terms of building a "supercomputer" for CFD on reasonable budget (5000 $)

thank you

 

[tommyk19]

I would start with a careful inventory of all production software
you intend to run on a regular basis.

Let me give you a good example from our daily workload:
after we installed COPERNIC desktop search software
on a quad-core Q6600 workstation, we compared it
head-to-head with a dual-core D 945 in a similar workstation.

The Q6600 finishes just about twice as fast as the D 945
updating the very same 5GB database.

That can only be the result of parallel programming in COPERNIC.

If your software is not coded to exploit multiple cores,
you're wasting your time bulking up on multi-socket systems.
Those are intended chiefly for busy servers with lots of
multi-tasking to process.

After you've completed your analysis of your production software,
the real choice for you is between a single-socket Core i7 machine,
or a twin-socket Xeon with Core i7 architecture.

A single-socket Core i7 machine has 4 CPU cores with hyperthreading:
each socket is THUS capable of running 8 threads simultaneously.

That should be enough for any software you want to throw at it,
particularly if you are only running one copy of your fluid dynamics model
at any given point in time.

Next decision is the clock speed of the Core i7: go for the fastest
because you've got plenty of budget for it, and it was just reported
on the Internet that all Intel Core i7 CPUs are "unlocked" --
meaning they can all be overclocked.

So, buy a motherboard that makes overclocking easy, e.g. ASUS P6T.

As for RAM, X58 chipsets now support either 12 or 24GB of DDR3 RAM:
look into Corsair's high-end Dominator series, with the memory module cooler.
Kingston make engineering samples of 4GB DIMMs x 6 = 24GB total e.g.:

http://www.hexus.net/content/item.php?item=17187


The Corsair power supplies w/ 850 Watts and up should be enough:
they are very highly regarded for high-end workstations like the
one you want to build:

http://www.newegg.com/Product/Product.aspx?Item=N82E16817139009&Tpk=N82E16817139009

I'll leave your graphics hardware choices up to your own good research.

If you are really serious about writing very large data files,
then be sure to give serious consideration to motherboards
and/or RAID controllers that can pump data quickly to and from
SAS (Serial Attached SCSI) hard drives: these come in both
10,000 and 15,000 rpm e.g. Seagate.

Despite what lots of amateurs claim, in ignorance,
a RAID 0 with multiple (4 x or 8 x) HDDs can really move a lot of raw data
very fast: 250-300MB/second is quite easy, and 500MB/second
is within reach, if you know what to buy and how to configure
your storage subsystem. The key is choosing a RAID controller
that does parity computations in hardware, e.g. Areca, 3Ware
or Highpoint's "enterprise" class controllers (there are others).

Out on the "bleeding" edge you will find Fusion-io's ioDrive Duo
(Google that one), or OCZ's new Z-drive (actually just a
Highpoint RocketRAID with 4 x OCZ MLC SSDs inside the
plastic shell).

Lastly, we've had enormous success with ramdisks
by installing RamDisk Plus from SuperSpeed, LLC
in Sudbury, Massachusetts: www.superspeed.com

A little bit of intelligent memory management, tailored
to your application software, can pay enormous dividends:
Core i7 memory bandwidth BEGINS at 25,000 MB/second
(25GB/second), and goes up from there when triple-channel
DDR3 is overclocked.


This is THE hottest computer hardware available
at the present time for workstation-class machines.
AMD still can't even come close.


hope this helps


MRFS

Hi there,

I have found this thread and really liked you reply. Question is if I ask same question now (mean 2012) what would be reply or what would you change and what would you keep on above mentioned setup?

I'm in similar situation so wondering where to start in terms of building a "supercomputer" for CFD on reasonable budget (5000 $)

thank you

 

[majestic1805]

Holy thread ressurection...

The thing with supercomputers is that they're a series of machines. It's not all one chasis. So, a home setup would likely be a series of midtower boxes and some software to distribute instructions across the cluster. SSDs would increase the throughput to really good levels especially if you have striped arrays.