Physics and scientific computing workstation build

[BenK1010]

Hello everyone,

I work in a university physics laboratory and we want to build a workstation to run our physics simulations on. The budget is up to $7k. The software we use utilizes all available cores - it has been used by others on clusters and servers with multiple CPUs and dozens of cores. We want a multicore, high-RAM design. A GPU is not necessary.
Physics processing is highly parallel so the more cores and threads supported the better.

We would like at least 2 processors of 6/8 cores and 64GB of fast RAM, with a motherboard such that the RAM can be expanded later if needed (as much as possible, but at least to 128 GB). The mobo should also support a GPU in case we get one later (it helps with 3D plots and animations of results, although it is optional).

Now, with a ton of RAM, I would think that RAM speed and bus width is important, so I guess we want dual or quad channel, 2000MHz or so. I have heard that, since Ivy Bridge, RAM speed is more important that latency but I don't know why. Can you clarify?

Additionally, I have heard that Intel processors don't support over 1333MHz RAM without overclocking. True?

So, to summarize:
A 2 processor (likely Intel Xeons), 64-128 GB RAM build with accommodations for expansion.
$7k budget
Things we do not need: monitor, DVD drive, GPU, wifi
And, do I need to overclock to get the best RAM performance?

What are your recommendations?
Thanks in advance.

 

[DaCoder]

You should check out some benchmarks that compare all processors together. A good site that does this is Passmark. (It also tells prices) However, I have no experience with OC and motherboards. Here is the link to Passmark: http://www.cpubenchmark.net/

 

[maestro0428]

Latency is more important that speed in my opinion. A CAS 7 kit at 1600mhz will be faster in most situations than 2400mhz CAS 10. Processors can get finicky when you are adding large amount of memory at those speeds too. Xeons will run faster memory IF the motherboard allows it. ASUS makes a good dual 2011 motherboard that is perfect for workstations. http://www.newegg.com/Product/Product.aspx?Item=N82E16813131817&IsVirtualParent=1 Its seems the BIOS is limited to 64GB of memory at this point...but it will support 2133mhz. Does your work support GPU acceleration? The work wonders and have a ton of compute power.

 

[bambiboom]

BenK1010,

My recommendation for a physics, simulation system for performance, error-correction, and high stability is 2X Xeon > ECC RAM > Tesla GPU coprocessor as follows >

BambiBoom CalcuCannon Cadasimumathematica Supermodeler ™℞™©®©$_ 7.2.13

1. (2) Intel Xeon E5-2687W Sandy Bridge-EP 3.1GHz (3.8GHz Turbo Boost) 20MB L3 Cache LGA 2011 150W 8-Core Server Processor $3,868. ($1,934 each) On the Passmark CPU benchmark chart, this processor is the No.2 rated, score = 21491. In the top 100 systems on Passmark Performance test, this CPU is used in the 6th highest performing system. If this CPU option is selected, the Tesla C2075 is deleted.

1A. 2X Xeon X5680 , six core 3.3 / 3.6GHz CPU's, new about $1,600 ($800 each) with new Tesla C2075 or used Tesla K20 (2,496 core coprocessor) for about $2,000. The X5680 is obsolete and previous generation, but available new and extremely good performing, in multiple CPU benchmarks, rated no. 14 at a score of 14151. As you can see, the CPU score is similar to E5-2640, but clock speed is higher.

1B.(2) Intel Xeon E5-2640 Sandy Bridge-EP 2.5GHz (3GHz Turbo Boost) 15MB L3 Cache LGA 2011 95W Six-Core Server Processor $1,824. ($ 914 each.) Rating No. 10, score = 14712

1C. (2) Intel Xeon E5-2643 3.3GHz LGA 2011 130W Quad-Core Server Processor for UCS C220 M3, C240 M3 UCS-CPU-E5-2643= $2,219.98 ($1,109.99 each) Rating No 14, score= 13655

2. (2) CORSAIR Hydro series H50 High Performance Liquid CPU Cooler ($120) ($59.99 ea) Liquid cooling is not typical for workstations, but having experience with long, sustained multiple CPU systems (CAD rendering which uses all cores), CPU temperatures can rise quickly to high levels.

3. SUPERMICRO MBD-X9DR3-F-O Extended ATX Server Motherboard Dual LGA 2011 DDR3 1600 $439.99 (High performance and 12 RAM slots support up to 512GB RAM. Supports ECC RAM at 1600 native. )

4. + 64GB Kingston (4 x 16GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Registered Server Memory DR x4 Model KVR16R11D4K4/64 $579.99 Note

5. NVIDIA TESLA 900-21030-0020-100 Tesla C2075 6GB 384-bit GDDR5 PCI Express 2.0 x16 Workstation Video Card $1,999.99 (The C2075, with 448 CUDA cores acts as GPU coprocessor / accelerator and has a supplementary single monitor output)

6. Samsung 840 Pro MZ-7PD512 512 GB 2.5" Internal Solid State Drive -$519.99 (Operating System (s) and Applications)

7. (2) Western Digital RE WD4000FYYZ 4TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Enterprise Internal Hard Drive $819. ($409.99 each) (RAID 1) (Active files, backup, system image)

8. SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply $254.99

9. LIAN LI PC-D8000 Black Aluminum ATX Full Tower Computer Case $331.99

_______________________________
TOTAL = about > $6,700 with Xeon X5680 + new C2075 or used K20, $6,900 with Xeon E5-2640, $7,000 with Xeon E5-2687W =minus C2075, $7,300 with E5-2643

Notes:

1. Your specification mentioned that a GPU was not needed, but I am curious as to the method of monitoring input and output. Note that Xeon processors do not have integrated graphics. If you find that you would need a graphics card for 2D only, I'd recommended the Quadro K600 for about $175. If you are running visualizations / animations of 3D models / simulations, then a Quadro K5000, about $1,800.

2. The above configuration was done in consideration of the best cost /performance basis within or near the budget and would be suitable for any high calculation density requirements > 2D and 3D dynamic CAD modeling and animation, NAMD molecular biology, static and dynamic structural, vector analysis, particle, fluid, and thermal simulation. without knowing specific applications, my inclination for your use would be for the 2X Xeon E5-2687w (= w/o Tesla C2075).

3. Note that in this kind of system, overclocking is likely to increase calculation errors and system instability, and is the reason that Xeon processors have locked multipliers and can not be overclocked. Also, ECC RAM runs at a maximum speed of 1600 and in any event, higher RAM speeds are not usually experiential. lower RAM latency however, may be experiential in your proposed uses and there may be some reason to consider using 8GB ECC RAM modules having a lower latency instead of 16GB.

4. In place of the Tesla C2075, it may worthwhile to consider other models of Tesla GPU coprocessor as used components, e.g., the C2070 which is also 448 CUDA cores and 6GB, but used is available for $1,200-1,400, the M2050 with 448 Cores can be $500-600, and I saw M1060's- 240 Cores for only $85 recently. Used K10's are sometimes seen for $1700 and K20's for $2,000. If you have the right application that can use these coprocessors, they are a key to near supercomputer calculation power.

Enhancements >

A. NVIDIA TESLA K20 (900-22081-2220-000) GK110 5GB 320-bit GDDR5 PCI Express 2.0 x16 3.52 Tflops Workstation Video Card $3,500 (This provides 2,496 CUDA co-processing cores. Used (18,866!) of the similar K20X in banks in the Oak Ridge "Titan" supercomputer, currently the most powerful computer in the World).

B. Kingston 64GB (4 x 16GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Registered Server Memory DR x4 Model KVR16R11D4K4/64 $579.99 (=Total RAM of 128GB) (With 8X 16GB slots filled =-128GB, subsequently another 64GB (4X 16GB) or (4X 32GB )can be eventually added for a total of 192GB or 356GB)

C. Western Digital RE WD4000FYYZ 4TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Enterprise Internal Hard Drive ($409.99) Highly recommended. Adding a 3rd storage Drive allows a RAID 5 in which two drives are put in RAID 0 striping for performance and the 3rd drive acts in RAID 1 mirroring to protect the system files and data backup. If any drive, OS/application or storage fails, the RAID 1 can restore from a stored system image and file backup.

Cheers,

BambiBoom

[Dell Precision T5400 > 2X Xeon X5460 quad core @ 3.16GHz, 16 GB ECC , Quadro FX 4800 (1.5GB), WD RE4 / Segt Barcd 500GB > Windows 7 Ult > AutoCad, Revit, Solidworks, Sketchup, Adobe CS MC, Corel Technical Designer, WP Office, MS Office (The above system has a rating of 1859, with a CPU score of 8458, 2D of 517, 3D of 1097)

 

[maestro0428]

A Tesla card may not be necessary. If you applications support CUDA, a GeForce will work just as well. You will loose ECC gpu memory going that route though. A 690 or Titan would be right up your alley.

 

[Lord_Kitty]

You might find a use for the Xeon Phi. Its a PCI-e coprocessor that you can use parallel processing on common languages. Its basically a CUDA workstation GPU but on an Intel architecture.

This page explains it all: http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-phi-coprocessor-brief.html


If your application truly can use 61 cores, 244 threads, then saving on the CPU and putting a larger budget on a Xeon Phi might be more beneficial.

 

[BenK1010]

Wow, bambiboom, you know your stuff. Thanks a lot!

I did say that a GPU is unnecessary, but had forgotten that the Xeons don't have integrated graphics. So I guess we'll need one. However, our primary software (Comsol Multiphysics) does not currently support Tesla/Cuda and the like (although Mathematica does), so we do not need a Tesla card. Any GPU will do - NVIDIA Quadro or AMD Firepro. Any recommendations here? It only needs to be a basic model.

Given that we aren't using Tesla anyway, should we stick with the Xeon E5-2687W even though it costs twice as much as the other options?
bambiboom, what do you think about the Xeon Phi? Although I don't know if Comsol supports it yet.

I've heard that closed-loop water coolers aren't so much better than air cooling. Is this true?

I also feel that a solid state drive is nice but unnecessary.
And good point on the RAID. We once had 2 disks fail on the same day. Unbelievable.
Is a separate RAID controller necessary?

Addendum also, it seems to me that water cooling carries the risk of the cooler springing a leak and ruining a very expensive computer...

 

[bambiboom]

BenK1010,

I'm very pleased to consider this kind of system as it requires me to extend my knowledge of relevant software and current hardware- and COMSOL Multiphysics is a new one- and as I'll explain- marvelous in the true sense of being deeply mystifying.

"I did say that a GPU is unnecessary, but had forgotten that the Xeons don't have integrated graphics. So I guess we'll need one. However, our primary software (Comsol Multiphysics) does not currently support Tesla/Cuda and the like (although Mathematica does), so we do not need a Tesla card. Any GPU will do - NVIDIA Quadro or AMD Firepro. Any recommendations here? It only needs to be a basic model."

GPU> In review of the specifications of COMSOL Multiphysics >

http://www.comsol.com/products/specifications/cad/

> input files include some of the most GPU demanding applications such as Solidworks, Inventor, and Catia, all of which are potentially CUDA accelerated >

http://www.nvidia.com/object/gpu-accelerated-applications.html
http://www.nvidia.com/object/gpu-accelerated-applications.html

And, specifically, the nature of the COMSOL Metaphysics output appears to be intensely dynamic vector and Boolean calculation intensive -meshes >

COMSOL Multiphysics

Electrical
AC/DC Module
RF Module
Wave Optics Module
MEMS Module
Plasma Module
Semiconductor
Module

Mechanical
Heat Transfer Module
Structural Mechanics Module
Nonlinear Structural
Materials Module
Geomechanics Module
Fatigue Module
Multibody Dynamics Module
Acoustics Module

Fluid
CFD Module
Microfluidics Module
Subsurface Flow Module
Pipe Flow Module
Molecular Flow Module

Chemical
Chemical Reaction
Engineering Module
Batteries & Fuel Cells
Module
Electrodeposition Module
Corrosion Module
Electrochemistry Module

LiveLink™ for Excel®
LiveLink™ for SolidWorks®
LiveLink™ for SpaceClaim®
LiveLink™ for Pro/ENGINEER®
LiveLink™ for Solid Edge®
LiveLink™ for Creo™ Parametric
LiveLink™ for AutoCAD®
LiveLink™ for Inventor®
File Import for CATIA®V5

Multipurpose
Optimization Module
Material Library
Particle Tracing Module

Interfacing
CAD Import Module
ECAD Import Module
LiveLink™ for MATLAB®

Consequently from the above, I'm mystified by the the COMSOL list of recommended grpahics cards >

Tested AMD Graphics Cards
Card Model Operating System Driver Version
FirePro™ V3900 Windows 7 8.911.3.1-120116a-132089E
FirePro™ V4900 Windows 7 8.85.7.1-110717a-124599C
FirePro™ V5900 Windows 7 8.85.7.2-110901a1-125182C
FirePro™ V7750 Windows Server 2008 R2 HPC 8.540.0.0
FirePro™ V7800 Windows 7 8.85.7.1-110717a-122786C
FirePro™ V7900 Windows 7 8.83.5.4-110520a-119550C
FirePro™ W5000 Windows 7 8.982.2
FirePro™ W7000 Windows 7 8.982.2

Tested NVIDIA® Graphics Cards

Card Model Operating System Driver Version
Quadro™ FX 3800 Windows Server 2008 R2 8.17.12.7614
Quadro™ FX 540 Windows XP 32-Bit 6.14.12.7614
Quadro™ FX 540 Windows XP 64-Bit 6.14.12.7565
Quadro™ FX 550 Windows Vista Pro 64 Bit 7.15.10.9746
Quadro™ FX 580 Windows 7 8.16.11.9175

There is simply a cognitive disconnect with the above list of graphics cards and the ability to open and run the likely input files in 3D which in addition will be running complex, multi-valent simulations that can include acoustics, structural, thermal, particle / gaseous, chemical, and etc. I have, in my third tier computer, a Quadro FX 580 (512MB and it will not navigate a 20MB model in Sketchup, never mind a jet engine combustion chamber simulation input from CATIA. And, the Quadro 550 is only 128MB. I recently tested a system (architectural office) using the Quadro 550 and which scored on Passmark Performance Test 8, a 74 in 3D. To put that in perspective, my two generation obsolete Quadro FX 4800 scored 1097 and that performance is nominal in Solidworks 2010. A couple of the Firepros, such as the W7000 and V7900 are more in line with my expectation, but I'm baffled!

You will be more familiar with the requirements of COMSOL than I, but given the extremes of the calculation performance expectation and the nature of potential input files, plus the expectation of high resolution rendering and simulation animation, I would suggest a system that was optimized for high performance in the most demanding input and output applications and which on the whole can take advantage of CUDA acceleration, i.e, along the lines of the previous recommendation.

If COMSOL needs massively parallel computational power >

""COMSOL supports shared-memory parallelism and distributed-memory parallelism on 32-bit and 64-bit Windows. Distributed-memory parallelism requires Windows Compute Cluster Server 2003 with Service Pack 1, Windows HPC Server 2008, or Windows HPC Server 2008 R2.""

> yet can run and display that range of simulations on a Quadro 550, I would be completely astounded.

"Given that we aren't using Tesla anyway, should we stick with the Xeon E5-2687W even though it costs twice as much as the other options?"

CPU > Again, the ability to use all available cores and the intensity of the calculations involved, the 16 cores / 32 threads at that clock speed of the dual E5-2687W would present almost the best configuration possible short of a 4 or 8 E7 Xeon 10-core system, which in any event would be using 4 or 8 CPU's that cost at least $3,500 each.

While I think you would have better performance from a specially assembled system, you can also simply buy a dual E5-2687w Dell Precision T7600 system with a Quadro 4000 >

http://www.ebay.com/itm/Dell-Precision-T7600-Dual-2X-Eight-Core-E5-2687W-64GB-Quadro-4000-PERC-H710P-/141006436053?pt=Desktop_PCs&hash=item20d4a376d5

> which has 64GB RAM for $5,800 (new $12,000). I would sell the Quadro 4000 for $350 and buy a Quadro K4000 (3GB) for $800, and add another 64GB RAM for another $600, so there's $6,850- nicely within your budget and ready to use right away.

"bambiboom, what do you think about the Xeon Phi? Although I don't know if Comsol supports it yet."

I've been reading about Xeon Phi , but a the moment it seems that only specially written C++ programs can take advantage whereas most the likely COMSOL input applications are already CUDA accelerated and even have special display drivers. This again assumes that the new system is not only running simulations on the input files but also has to navigate and input parametrics, limits, display and produce animation and renderings, and so on.

"I've heard that closed-loop water coolers aren't so much better than air cooling. Is this true?"

I have never used liquid cooling and probably would prefer good fan / heatsink cooling. However, just yesterday I ran a fairly simple, small scale rendering using 12 of the 16 threads on my 8 core / 16 thread system and I saw RAM temperatures of 93C and the CPU cores were all 85-89C within 12 minutes. So perhaps I was overreacting a bit based on my mild panic yesterday! Yes, you could use a pair of good PWM fan / heatsinks and though I've never used them, I 'm inclined by the size and radiator area to think of >

(2) Noctua NH-U14S 140x150x25 ( NF-A15 PWM) SSO2-Bearing ( Self-stabilising oil-presure bearing ) CPU Cooler $152 ($76 each)

http://www.newegg.com/Product/Product.aspx?Item=N82E16835608041

"I also feel that a solid state drive is nice but unnecessary."

It's true that SSD's are mostly beneficial in OS and application loading / startup and I would gladly exchange the SSD for the OS / applications to be a RAID 5 or 10 that has both performance RAID 0 and mirroring RAID 1. As long as they are 6GB/s and 64MB cache, those could be even 500GB disks. >

Western Digital WD VelociRaptor WD5000HHTZ 500GB 10000 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive - OEM $150 each

These do not also necessarily have to be 10K RPM drives but could be a pile of >

Western Digital WD Black WD5003AZEX 500GB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive $75 each.

The reason I like 500 GB drives if sufficient, is that they are likely to be single platter and will be much more reliable. I've used a WD RE4 500GB that has been running almost continuously for over 4 years with no errors or odd behaviour.

"And good point on the RAID. We once had 2 disks fail on the same day. Unbelievable."

Yes, RAID has a reason and given the nature of your work and amount of preparation required, I would leave nothing to chance. I would advocate to always keep a full system image- made immediately after configuration and before use in a RAID 1 region so that the OS and applications can be instantly restored in case of a HD failure and because that image is pristine, occasionally restoring just to refresh and remove accumulated registry errors.

Oh yes, and if it will be used as the graphics card for this system, I will be pleased to donate an excellent, never overheated Quadro 550.

Cheers,

BambiBoom

"Never use one word where twenty will do just as well."

 

[BenK1010]

We are actually using Comsol without a GPU now on a desktop. You are right that Comsol can accept CAD files, or the user can build the geometry directly in Comsol, but our geometries are quite simple (we mostly simulate nanoparticles). So it is possible to have a simple geometry that is not GPU intensive and complex physics that is very CPU intensive. Of course, we might be using more complex geometries at some point so a GPU would be most welcome.

Now, we might be using this workstation for heavy Matlab work and Matlab supports Tesla/CUDA, but the primary tool is still Comsol, so it's probably not worth buying a Tesla. Also, Comsol does not support CUDA but might offer support for the Xeon Phi at some point, so while I don't think we should spend on either the Phi or the Tesla now, the motherboard should allow for their addition later.

According to the Newegg page for that motherboard it does not support 150W processors, which the Xeon E5 2687 is. It also says (in a review there) Kingston memory does not work with this board. Can you suggest another motherboard? I also cannot often find out whether a mobo is 1.5v or not. Anyway the mobo needs: A lot of RAM (up to 256-512GB),in quad channel, dual sockets, PCI-e socket for GPU, and support for RAID, Xeon Phi, and Tesla in case we add the latter two items later. It also needs to be 1.5v since Intel processors aren't designed for higher voltages.

Interestingly, the Xeon x5680 is selling for $1,600 on newegg. Quite pricey for an "obsolete" CPU.

If I'm going to buy, I'd rather purchase from a vendor that specializes in HPC such as Colfax. But then it will cost a lot more.

 

[bambiboom]

BenK1010,

Motherboard > I had chosen the Supermicro motherboard on the basis that in several cases it placed very high in Passmark benchmark baselines using 2X Xeon 2687w, although the highest rated Xeon 2687w system was an HP. I checked Passmark and other motherboard that did very well included the ASUS Z9PE-D8 WS Dual LGA 2011 Intel C602 SATA 6Gb/s USB 3.0 SSI EEB Intel Motherboard ( $580). However, this board supports only up to 64GB - but that also says in the small print, "supports up to 135W". I like Tyan boards, and they support more RAM, but they also had a 135W per CPU limitation. Still, all the above boards are shown in Passmark baselines as top performers with a pair of 2687's, so a number of users are ignoring the specifications.

I found what I think are three strong possibilities >

1> INTEL S2600COESSI EEB Server Motherboard Dual LGA 2011 DDR3 1600 ($550-$620). (Supports up to 512GB DDR3 ECC UDIMM/LV)

i. http://www.intel.com/content/www/us/en/motherboards/server-motherboards/server-board-s2600co.html
ii. http://ark.intel.com/products/63157
iii. Compatible Memory validation> http://www.intel.com/content/www/us/en/platform-memory/ddr3-udimm-ecc-e5-family-memory-list.html
iv. Review 2012 > http://www.pugetsystems.com/labs/articles/Product-Review-Intel-S2600COE-144

Intel S2600COE > 16 DIMM/ ECC up to 1600 up to 512GB, 84 PCIe lanes > 4X PCI Express 3.0 x16, 1 XPCI Express x8, 1 x PCIe x8 Gen 3 1X PCI Express x4. 1 x PCIe x4 Gen 2 that would accommodate a a GPU plus Tesla or Xeon Phi coprocessors and there will also be room for something like an LSI Logic RAID controller and/or RevoDrives. 10 SATA ports, LSI RAID support, 4 Ethernet /LAN > every conceivable technology for everything- very complete specifications. This has among the widest maximum memory bandwidths I've ever seen- 102GB/s which is perfect as 2X Xeon E5-2687w has 2X 51GB/s. A dual Xeon X5680 system would have a maximum bandwidth of 64 GB/s (2X 32GB/s). That's of course the great feature of dual CPU's- it creates so many more PCIe lanes and memory controllers. Above is a link to the compatible memory validation for that platform, but these are from 2012 and mysteriously no larger than 8GB modules. Some more research needs to be done to find by brand and model the specific RAM tested on the Intel S2600COE. If you think this board is a good choice, I can have a look at memory. Also, Intel boards have just fantastic build quality.

The review linked above is very informative and notes that this board- and several others I've looked at, have non-standard mounting holes. That review also mentions that this board is more server oriented than workstation and suggests instead using the ASUS Z9PE-D8 WS- which specifies CPU's up to 135W,...

I'm not certain- they don't identify the components used, but I had a look at the Colfax workstation solutions and S2600COE may well be the board they're using. The Colfax site has very good information on the SOTA of Xeon Phi, and while more broadly applicable than I thought, it appears to be oriented towards sophisticated custom code applications, whereas CUDA coprocessing is oriented towards mainstream applications that can use it. This is an argument for Xeon Phi for your use.

2> SUPERMICRO MBD-X9DAi-O Extended ATX Server Motherboard Dual LGA 2011 DDR3 > $490

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182260

http://www.supermicro.com/products/motherboard/xeon/c600/x9dai.cfm

16GB ECC registered 1600 compatible memory > http://www.supermicro.com/support/resources/memory/display.cfm?sz=16.0&mspd=1.6&mtyp=44&id=8939EE74FDFB9E5DB353166AAAE42C35&prid=82361&type=DDR3&ecc=1&reg=1&fbd=0

A 16GB Samsung module from the compatible list > http://www.newegg.com/Product/Product.aspx?Item=N82E16820147183&Tpk=Samsung%20M393B2G70BH0-CK0&IsVirtualParent=1 ($180 each X4= $770 for 64GB, 8X= $1,440 for 128GB)

This board also uses the Intel C602 chipset, supports 150w CPU's and has 16 slots / 512GB RAM, and has 3x PCI-E 3.0 x16 slots, 2x PCI-E 3.0 x8 slots, 1x PCI-E 3.0 x4 (in x8) slot*** . This board does seems to have very good features and produces very good ratings on Passmark and the build quality is very good, though I think not quite Intel standard.

***[ The Intel S2600COE has 4X PCI-E 3.0 x16 slots which may be useful in the event multiple Xeon Phi coprocessors are added. I feel that a RAID controller is probably called for, so a future configuration could be > Slot 1 = Quadro GPU, S2= Revodrive 3 X2, S3=LSI Logic or similar RAID controller, S4= Xeon Phi Coprocessor. Of course, if a Revodrive or other PCIe drive, I call these "direct injection" drives, is not used, then there could be 2X Xeon Phi (about $10,500).]

3> ASUS Z9PE-D16 SSI EEB Server Motherboard Dual LGA 2011 DDR3 1600 > $480

http://www.asus.com/Commercial_Servers_Workstations/Z9PED16/#specifications

memory > http://dlcdnet.asus.com/pub/ASUS/mb/Socket2011/Z9PE-D16/Manual/Z9PE-D16_Series_Memory_AVL_20130610.pdf

Intel C602-A PCH, 16 DIMM, supporting 512GB ECC 1600, Total Slots : 6 > PCI-E x16 3.0 with Slot 5 running at x8 if Slot 6 is occupied, and Slot 6 is MIO

This appears to be another very good choice and has even better flexibility in terms of the number of PCIe x16 slots.

Compatible memory for the ASUS Z9PE-D16 > SAMSUNG- 16GB (1X16GB) 1600MHZ PC3-12800 CL11 DUAL RANK ECC REGISTERED DDR3 SDRAM DIMM GENUINE SAMSUNG SERVER MEMORY (M393B2G70BH0-CK0) > 128GB= $1,160 ($145 each, Server Direct)

GPU> The S2600COE has onboard graphics but that must share and therefore divert system RAM. I think it's worthwhile to have a good graphics card on hand that could handle high resolution graphs, waveforms, visualization models, and animations. As you mention the geometries encountered are quite simple, for most work it would seem the Quadro K600 would be sufficient. Autodesk certifies the K600 for both Maya and Inventor 2013 which has extensive simulation capabilities. With single and double precision, error correcting RAM, and error correcting VRAM in the GPU, this system will be capable of high accuracy 3D models of particle phenomena like smoke, cosmic dust, comet, asteroid trajectory, extra-solar planetary gravitational perturbation, or galactic evolutionary models. Or structures > crystalline, cellular, vector phenomena, metallurgical, 3D particle annihilation models / vector diagrams. As rendering is CPU-based, this system would be extremely rendering capable. The potential capabilities and expansion of future uses would be significantly enhanced by using, optionally, a Quadro K2000 (2GB) $420, which has much better 3D modeling performance.

The Xeon X5680 > I'm surprised also that the X5680 (released Q1, 2010) can be still so costly- they were originally $1,660, but it does stand out as one of the still very desirable Xeons as it is 6-core and among the higher Xeon clock speeds. If I were building a system for myself today (industrial design), I would probably use a single Xeon E5-1650 (6- core, 3.2 /3.8GHz, about $600), but if it were a dual CPU, I would use the X5680.

"Physics and scientific computing workstation build" > Preliminary Concept 2 >

BambiBoom CalcuCannon Simumathematica Supermodeler II™?™©®©$_ REV 7.4.13

The following configuration was done in consideration of the best cost /performance basis suitable for high calculation density requirements > Mathematical, Astrophysical, vector analysis, particle, fluid, gas flow, and thermal simulation, statistical analysis, molecular biology, static and dynamic structural, 2D/ 3D CAD modeling, graphic design, rendering, architecture, industrial design, financial analysis. >

1. (2) Intel Xeon E5-2687W Sandy Bridge-EP 3.1GHz (3.8GHz Turbo Boost) 20MB L3 Cache LGA 2011 150W 8-Core Server Processor > $3,868. ($1,934 each) On the Passmark CPU benchmark chart, this processor is the No.2 rated, score = 21491. In the top 100 systems on Passmark Performance test, this CPU is used in the 6th highest performing system.

2. (2) Noctua NH-U12S 120x120x25 ( NF-F12 PWM) SSO2-Bearing ( Self-stabilizing oil-pressure bearing ) CPU Cooler > $140 ($70 each)

3. INTEL S2600COESSI EEB Server Motherboard Dual LGA 2011 DDR3 1600 > $580 (Newegg). (Supports up to 512GB DDR3 ECC UDIMM/LV) (see links above for information)

3A. ASUS Z9PE-D16 SSI EEB Server Motherboard Dual LGA 2011 DDR3 1600 > $480 (See links above for information)

4. 128GB (8 X 16GB) DDR3- 1600 ECC RAM for Intel S2600COESSI EEB > About $1,200 (compatibility to be researched)

4A. 128GB (8 X 16GB) SAMSUNG M393B2G70BH0-CK0 1X 16GB for ASUS Z9PE-D16 > 128GB= $1,160 ($145 each, Server Direct)

5. NVIDIA Quadro K600 (with Intel S2600COESSI EEB) VCQK600-PB 1GB GDDR3 PCI Express 2.0 x16 Low Profile Workstation Video Card >$160

5B. NVIDIA Quadro K2000 Video Card (used with ASUS Z9PE-D16 option)- 2GB GDDR5, PCI-Express 2.0 (x16), 1x Dual-Link DVI-I, 2x DisplayPort, DirectX 11, Single-Link, Fan, (VCQK2000-PB) $420.

6. Western Digital WD Black WD1002FAEX 1TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive $90 (Operating System (s) and Applications) While an SSD would load the OS and applications noticeably faster, and large file transfers would be faster, the general experiential qualities of using a 6GB/s mech'l drive should not be a problem.

7. (3) Western Digital WD Black WD2002FAEX 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive - OEM $480 ($160 each) (RAID 5) (Active files, backup, system image)

8. SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply > $254.99

9. LIAN LI PC-A75 Black Aluminum ATX Full Tower Computer Case $182

10. ASUS DVD Burner 24X DVDDRW-24B3ST/BLK/G/AS $28.

_______________________________
TOTAL = Configuration A > Intel S2600COE / Quadro K600 about > $6,975 (depending on RAM cost) OR Intel S2600COE / Quadro K2000 about > $7,335 (depending on RAM cost) OR Configuration B with ASUS Z9PE-D16/ Quadro K2000 > about $7,103

Given your comments, these are the best cost/performance solutions within /close to the budget.

Notes to the above Specification >

A. If it's not proprietary, it would help to have a quite specific description of a typical task for this system so as to make certain it can be optimized.

B. A DVD-R drive is added if only to load OS, network software, and programs, and make spot backup and transfer disks.

C. If the ASUS Z9PE-D16 is determined to have no disadvantages, it's slightly lower cost and allowing the budget overrun to about $7,103 will present a much enhanced 3D capability, that I would recommend for future expansion of use of this system.

D. If the budget allows, I would rather use for the system enclosure > LIAN LI PC-D8000 Black Aluminum ATX Full Tower Computer Case $331.99 (= + $150) >

http://www.newegg.com/Product/Product.aspx?Item=N82E16811112390&Tpk=LIAN%20LI%20PC-D8000&IsVirtualParent=1

> as it provides excellent working space, air flow, and drive expansion possibilities. As well, again if there's a budget for it, I would upgrade the power supply to a server quality and possibly have redundant supplies, which that case would accommodate.

E. A small detail, but this system would benefit from a temperature monitor, fan control panel. Perhaps >

NZXT Sentry-2 5.25" Touch Screen Fan Controller $28.
http://www.newegg.com/Product/Product.aspx?Item=N82E16811992005

F. Additional case and dedicated RAM cooling fans should be considered.
_____________________________________

I couldn't resist considering whether it's possible to derive a co-processing configuration close to the $7,000 budget. In this instance, a Xeon Phi was out of the question as each module is $5,000. However, it is possible with a Tesla K20>

BambiBoom CalcuCannon Simumathematica SuperTesloidica III™?™©®©$ 7.5.13

1. Xeon E5-1650 6-core 3.2 /3.8GHz, 12MB cache, LGA 2011 $600

2. Noctua NH-U12S 120x120x25 ( NF-F12 PWM) SSO2-Bearing ( Self-stabilizing oil-pressure bearing ) CPU Cooler > $70


3. ASUS P9X79-E WS LGA 2011 Intel X79 SATA 6Gb/s USB 3.0 SSI CEB Intel Motherboard $499.99

4. 64 GB SAMSUNG 8GB 240-Pin DDR3 SDRAM DDR3 1600 ECC Registered Server Memory Model M393B1K70DH0-CK008/Q8/E8 $719.92 ($89.99 ea)

5. NVIDIA TESLA K20 (900-22081-2221-000) GK110 5GB GDDR5 PCI Express 2.0 x16 SLI Supported Workstation Video Card $3,499.99

6. NVIDIA Quadro K2000 VCQK2000-PB 2GB GDDR5 PCI Express 2.0 x16 Workstation Video Card $430.

7. Western Digital WD Black WD1002FAEX 1TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive $89.99 (Operating system / Applications)

8. (2) Western Digital WD Black WD2002FAEX 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive > $318. ($159.99 each)

9. SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply New 4th Gen CPU Certified Haswell Ready $254.99

10. LIAN LI PC-D8000 Black Aluminum ATX Full Tower Computer Case
Item #: N82E16811112390 $331

Total = about $6,900.

This is a very unrefined concept, and I can't predict the results, but at least a multi-GPU coprocessor configuration is possible at reasonable cost. This also opens the question as to whether double precision calculation density is higher when the CPU has fewer cores, but at a higher speed, and then running in parallel with 2,496 CUDA cores at about 700MHz. As massively parallel Tesla K20X's form the processing power of the Oak Ridge Titan, they must be doing something right!

From Newegg >

>""TPeak double precision floating point performance (board): 1.17 teraflops
Peak single precision floating point performance (board): 3.52 teraflops
Memory bandwidth for board (ECC off): 208 GBytes/sec [This is almost double the 2X Xeon 2687W's]

CFD, CAE, financial computing, computational chemistry and physics, data analytics, satellite imaging, weather modeling
SMX, Dynamic Parallelism, Hyper-Q.

Designed to be the performance leader in double precision applications and the broader supercomputing market,the Tesla K20 and GPU Accelerators deliver 10x the performance of a single CPU2.

Tesla K20 feature a single GK110 Kepler GPU that includes the Dynamic Parallelism and Hyper-Q features.

With more than one teraflop peak double precision performance, these GPU accelerators are ideal for the most aggressive high-performance computing workloads including climate and weather modeling, CFD, CAE, computational physics, biochemistry simulations, and computational finance."" <

I think this new realm of the Personal Supercomputer deserves some study- it writes a very different performance equation.
______________________________________________________

Bizarre Conjectural System Alternative 1 > As I'm not completely certain the tasks that will be assigned to the proposed system, I had a thought that could possibly yield greater total computing power within the budget- that is, in place of 16 cores / 32 Threads @ 3.1 / 3.8 GHz, there would be 24 cores /48 threads running at 3.33 GHz. For $1,400-$1,700 you can buy a used Dell Precision T7500 with one X5680, a reasonable one generation obsolete Quadro and at least 16GB of RAM/ With careful shopping add a second used X5680 for about $900 ($700 CPU, $200 CPU/ memory / fan riser) and new SSD and SAS drives. With a dual CPU, a Precision T7500 can accommodate 192GB DDR-3 1333 ECC RAM. So, for your $7,000 you could have 2 such systems with a Quadro 4000 and 64GB RAM each. Comparatively, the E5-2687w is ranked No. 2 , scoring 21521, and the 2X Xeon X5680 rated 14, scored 14151 for a total CPU power of 28302 , i.e. the 2X dual X5680 systems would present a capability of 131% of the single dual E5-2687w. The key is whether two machines are useful in separate, coordinated functions- that is, can work independently parallel on different aspects of a single problem that may be synchronized. This craziness arises from the description of COMSOL Multiphysics and recent supercomputers using modular CPU/ GPU coprocessing , like the Oak Ridge "Titan". If those independent modules (18,600+ AMD FX 8350 8-core CPU's each controlling a Tesla K20X, configured in rack-mount chassis like servers) can be linked in parallel there must be a sufficiently wide bandwidth connection existing.

Bizarre Conjectural System Alternative 2 > If the above idea has any validity, it can be improved upon- made more silly, by building 3 of the following, thus presenting a capability of 18 cores / 36 threads @ 3.2 / 3.8 GHz. Using a single six core Xeon E5-1650, each system would have a Passmark CPU rating of No. 12, CPU score of 11462 each for a total processing power of 34386 or 160% of the single dual E5-2687w system. >

BambiBoom PixelDozer Cadaedimathagrapharific ExtremeSignature VI REV A ®©™®™©™_6.30.13

This system is intended as a workstation solution with high performance / high precision, very high stability / reliability at reasonable cost, and suitable for 2D and 3D CAD, graphic design, rendering, mathematics, simulation, animation, and video editing.

1. Xeon E5-1650 6-core 3.2 /3.8GHz, 12MB cache, LGA 2011 $600 (Passmark CPU score= 11462, rank = No. 12)

2. Noctua NH-U12S 120x120x25 ( NF-F12 PWM) SSO2-Bearing ( Self-stabilizing oil-pressure bearing ) CPU Cooler $70

3. ASUS P9X79 WS LGA 2011 Intel X79 SATA 6Gb/s USB 3.0 SSI CEB $380.

4. 32GB (4X 8GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Unbuffered Server Memory > $300. (Check ASUS motherboard compatibility list)

5. NVIDIA Quadro K600 VCQK600-PB 1GB GDDR3 PCI Express 2.0 x16 Low Profile Workstation Video Card $160

6. SAMSUNG 840 Pro Series MZ-7PD128BW 2.5" 128GB SATA III MLC Internal Solid State Drive (SSD) $134.99 (OS and Applications)

7. (2) Western Digital WD Black WD1002FAEX 1TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive ($180) $90 (Files, Backup, System Image)

8. SeaSonic X Series X650 Gold ((SS-650KM Active PFC F3)) 650W ATX12V V2.3/EPS 12V V2.91 SLI Ready CrossFire Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply $120.

9. LIAN LI PC-A75 Black Aluminum ATX Full Tower Computer Case $182

10. ASUS DRW-24B1ST/BLK/B/AS Black SATA 24X DVD Burner - Bulk - OEM $17.

TOTAL > $2,144. or for three systems > $6,432.

___________________________________________________

The above system, with the substitution of a Quadro K4000, and three storage drives in RAID 5, and Windows 7 Professional was a concept for a system I would build for myself for industrial design work. I think however, I will wait for the next generation, 14nm Xeons and DDR4,... That series "Skylake" by the way is said to include 12 and 15 core CPU's!

As long as I'm entering the longest post in Tom's forum history competition, I may as well post a revised version of what I would consider a reasonable cost /performance, high performance system, also based on the Xeon 2687W, but with Tesla coprocessing- or which could be Xeon Phi (= + about $2,000). If you had specified a more or less unlimited budget, i.e. performance configured rather than cost /performance configured, for your proposed system, this would my suggestion >

BambiBoom CalcuCannon Cadaeditusimumathematica Extremesignature Modelistico $$+ IV ™®©$?™©_ 6.30.13 / REV a 7.3.13

The following configuration was done in consideration of the best performance and highest reliability basis suitable for the most demanding high calculation density requirements > Mathematical, Astrophysical, 2D and 3D dynamic CAD modeling and animation, molecular biology, static and dynamic structural, vector analysis, animation, particle, fluid, mechanical, and thermal simulation, rendering, graphic design > everything!

1. (2) Intel Xeon E5-2687W Sandy Bridge-EP 3.1GHz (3.8GHz Turbo Boost) 20MB L3 Cache LGA 2011 150W 8-Core Server Processor $3,868. ($1,934 each) On the Passmark CPU benchmark chart, this processor is the No.2 rated, score = 21491. In the top 100 systems on Passmark Performance test, this CPU is used in the 6th highest performing system (with Quadro 6000 (6GB).

2. (2) CORSAIR Hydro series H50 High Performance Liquid CPU Cooler ($120) ($59.99 ea) Liquid cooling is not typical for workstations, but having experience with long, sustained multiple CPU systems (CAD rendering which uses all cores), CPU temperatures can rise quickly to high levels.

3. Intel S2600COE SSI EEB Server Motherboard Dual LGA 2011 DDR3 1600 > $580. (Supports up to 512GB DDR3 ECC UDIMM/LV)

i. http://www.newegg.com/Product/Product.aspx?Item=N82E16813121589
ii. http://ark.intel.com/products/63157
iii. Manual >
iv. Compatible Memory > http://www.intel.com/content/www/us/en/platform-memory/ddr3-udimm-ecc-e5-family-memory-list.html
v. Review 2012 > http://www.pugetsystems.com/labs/articles/Product-Review-Intel-S2600COE-144

4. 256GB RAM (16x 16GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Registered Server Memory about $2700. (Check Intel motherboard / memory compatibility list)

5. PNY Quadro K5000 VCQK5000-PB Video Card - 4GB GDDR5, PCI-Express 2.0(x16), 1x Dual-Link DVI-D, 1x Dual-Link DVI-I, 2x DisplayPort, DirectX 11, Dual-Slot, Fan $1803.

6. NVIDIA Tesla K20 CUDA Coprocessor - 5GB GDDR5, PCI-Express 3.0 (x16), SMX, Dynamic Parallelism, Hyper-Q, Dual-Slot, Fan, $3261.

7. OCZ RVD3-FHPX4-480G RevoDrive 3 Series PCI-E Solid State Drive - 480GB, 4x PCI-Express $822. (Operating System (s) and Applications)

8. (3) Western Digital RE WD4000FYYZ 4TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Enterprise Internal Hard Drive $1230. ($409.99 each) (RAID 5) (Active files, backup, system image)

9. SeaSonic X-1250 1250W ATX12V / EPS12V SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply $254.99

10. LIAN LI PC-D8000 Black Aluminum ATX Full Tower Computer Case $331.99

_______________________________
TOTAL = about $14970, with Xeon Phi = about $17,000

_______________________________

Whew! I wonder if Tom's offers any "badges" for ridiculously long posts?


Cheers,

BambiBoom

My personal motto > "No matter your wealth, power, or friends, the cheapest things in life are free."

 

[maestro0428]

Man, that would make a bitchin' folding@home machine. The only thing I am unsure about it the gpu. A Quadro will get you ECC memory and 4GB and if you need that, then go for it, if not, bump down to a Geforce of at least a 560gtx. You could test run your calculations and if necessary, then pony up the extra 500-600 bucks for the K4000. Solid psu. Awesome SSD. I think your set.

 

[BenK1010]

Bambiboom, I love the names you give your computers!

Now, about the motherboard power issue, let's not forget that the Xeons other than the 2687W use 135 W or less so it would be easier to select a motherboard for those. They are also cheaper, except for the X5680 which is still very pricey as mentioned (unless used). But again the E5-2687 is superior and as this is all about CPU muscle, the CPU is the wrong place to save money. I considered the E5-2640 and E5-2643, which cost half as much, but they also have 12 and 8 threads and 6 and 4 cores respectively versus the 2687's 16 threads and 8 cores. I would go with the 2687 but if the lab director can't afford them, I will select the 2643 because it also has a high clock speed and bandwidth and also supports 1600MHz memory. The 2640 has lower clock speed, bandwidth, and supported memory speed. The X5680 is not sufficiently cheaper to justify purchasing it over the 2687 if budget is a problem, and it also uses a previous generation socket type, meaning that upgrading the CPU if we select this would entail replacing the motherboard, whereas the LGA 2011 is likely to be around for a while.

As for memory, both Samsung and Kingston 8GB and 16GB models are supported by the ASUS board and have comparable price. The thing is, this board supports quad channel memory and the Xeon 2687 supports 52GB/s bandwidth, which, for a 1600MHz DIMM is only achievable in quad channel memory configuration. So to really get the most out of the CPU, I'll need 4 or 8 DIMMs per CPU. I think I'll get 8x16GB = 128GB, leaving room for expansion to 256GB.

This ASUS motherboard supports RAID 5 for Windows, but not Linux (we want Linux). So I guess that means a hardware RAID controller will be necessary for RAID 5. Or just use RAID 1, which would be cheaper than buying a controller if we're not using a lot of discs.

A separate disc for the operating system and applications is a good idea but unnecessary.

Also, I don't see how the choice of motherboard should affect your choice of GPU here. All these motherboards can support any PCI-E GPU. And since we don't currently work with complex model geometries or do any graphics work, any basic GPU will do. The Quadro k600 will be fine, unless your offer to donate your old Quadro 550 still stands.

I feel pretty good about this solution. I still have to run it by my colleagues and of course may not get everything I want...but the BambiBoom CalcuCannon is the way to go. Thank you so much for your help, much appreciated!

 

[bambiboom]

BenK1010,

I'm not clear as to which motherboard on which you've decided, but the interesting aspect of the motherboard power rating was that there was so much evidence on Passmark baselines (results of Performance Test 8.0) of those using 2X E5-2687W on motherboards that were, as you pointed out, not rated for that use. Given the use for which that CPU was designed, and the $1,900+ cost, I'd imagine system builds would be more careful. To my fault, based on those results, I didn't verify that specification before recommending those couple of boards either.

The more I looked into the proposed CPU's the more I became convinced of the value of the E5-2687W- their performance advantage if all the threads can be employed is significant. The only CPU's that could improve on calculation density would be 10-core E7-4XXX and E7-8XXX that can be used in 4 and 8-CPU configurations and there are 8X CPU boards that can support 4,096GB of RAM. The problem of course is cost> the CPU's will be $3,500 to $5,000 each and 4TB of RAM requires 128X 32GB modules- and which cost about $900 each, so the budget would have to at least $200,000.

Yes, in my list, the 8X16 or rather 2X (4X 16GB) was used as the best use of the RAM and allow for future expansion. [ Historical Diversion > This reminds me of upgrading my first computer, a 1993 IBM 486 DX2 50MHz- (DX2 meant math coprocessor necessary for floating point applications like CAD). A 66MHz was the top end and cost $800 more. I added 2MB of RAM to have the maximum 4MB and which cost $180 or $90 per MB. At that rate, and this is the terrifyingly wonderful punch line to the story- 128GB of RAM would cost > 128 (GB) X 1024 (= 131,072MB) X $90 = $11,796,480! I also changed the 85MB HD for a 540MB HD at $1.25 per MB meaning 1TB = $1,280.] The golden age of computing is now!

I'm not entirely sure, but I think that all the E5 Xeons support 52GB/s memory bandwidth. This sets them apart from the E3's that are 21GB's and i7's, which are 25.6GB/s- exactly .5 . X5680's are 32GB/s. Another good reason to use E5. You're absolutely correct too that the LGA 2011 has more of future than the 1366 for the X5680.

It's difficult to evaluate the needs for drive performance. The idea of the separate OS / applications disk was so that the read /write system cycles were separate from the data R/W cycles, but an advantage depends on file sizes. If the system is set loose on long simulations, does it make periodic progress state backups? You mentioned a possible RAID controller and this was a component I'd considered originally as this system was looking more and more as requiring server functions. A RAID 5 is a very attractive idea as it gives the striping performance with the mirroring protection. With a system image installed on the RAID 1, any one disk could fail and the system could be entirely restored in a short time.

If a RAID controller is a possibility, avoid the Marvell controller at all costs- they are by reputation death on wheels, sluggish and unreliable. I would recommend a high-performance and well-developed and super-supported one, the "MegaRAID" Series by LSI Logic >

http://www.lsi.com/products/storagecomponents/Pages/6GBSATA_SASRAIDCards.aspx

I have an older series 4 port ( LSI SAS3080X) for the PCI-X bus that I haven't yet installed, but it is beautifully made, by specification and test, high performance (many top benchmark systems in Passmark use thseand, the users of these I know say it seems enthusiastically supported. There are drivers for several flavors of Linux-the 9261 supports Scientific Linux 5.5.

When you mention that the motherboard choice affected the GPU choices, my choices changed according to a very simple criterium- what's the best card that would fit in the budget. And this idea of best possible was due to imagining that there was 3D modeling of particle geometry. But, as your current system does not have a dedicated graphics card, I'll take your word that the demands are very light. I would however mention that onboard MB graphics operate- however slightly- to the detriment to CPU and RAM performance, so some card, no matter how modest will help.

Speaking of modest cards, my offer of the Quadro FX 550 stands. This has very low hours (not used for more than three years) in an architectural office. The system (Dell Precision 390) was dual core and in clean and low heat conditions. If you send me by private forum message your email and mailing address, I'll send it along. I can't think of a more exalted use for a 128MB card.

Cheers,

BambiBoom

 

[COCAYven]

Hello everyone,

I work in a university physics laboratory and we want to build a workstation to run our physics simulations on. The budget is up to $7k. The software we use utilizes all available cores - it has been used by others on clusters and servers with multiple CPUs and dozens of cores. We want a multicore, high-RAM design. A GPU is not necessary.
Physics processing is highly parallel so the more cores and threads supported the better.

We would like at least 2 processors of 6/8 cores and 64GB of fast RAM, with a motherboard such that the RAM can be expanded later if needed (as much as possible, but at least to 128 GB). The mobo should also support a GPU in case we get one later (it helps with 3D plots and animations of results, although it is optional).

Now, with a ton of RAM, I would think that RAM speed and bus width is important, so I guess we want dual or quad channel, 2000MHz or so. I have heard that, since Ivy Bridge, RAM speed is more important that latency but I don't know why. Can you clarify?

Additionally, I have heard that Intel processors don't support over 1333MHz RAM without overclocking. True?

So, to summarize:
A 2 processor (likely Intel Xeons), 64-128 GB RAM build with accommodations for expansion.
$7k budget
Things we do not need: monitor, DVD drive, GPU, wifi
And, do I need to overclock to get the best RAM performance?

What are your recommendations?
Thanks in advance.

Buy a Dell precision T5600 or T7600 if you wanna go up to colossal ram .
You will need ecc ram and xeons do run on 1600mhz ram.
If You need more than two socket solutions then you should instead move to amd set ups with opterons on quad cpu motherboards.
However if You can wait until 4th quarter of thsi year You will be able to buy the same workstations with New gen Xeons who will have a 30 mb cache with up to 12 cores, in few words you will run more than half teraflop in Double Precision on each cpu, and they will perform 3 times faster and heavier calculation than current xeons.
Focus on yoru cpus cache rather than clocks because it is all about how much information can be processed in one time rather than how fast you can calculate 2+2

If You was mentioning overclocking then Your probably not so good student in Your lab but spend more of Yoor time looking at gaming reviews on You tube.

Dell precision t7600 workstation with two xeons e5 2687W, will accomodate 128 gygas of ram 1600 mhz ECC.
You will need an OCZ revodrive3 X2 480/960 gygas boot PCIe SSD to manage all the ram and to configure Your windows to use less possible virtual ram on the system so to compute completely from the memory on the ram.
*MAKE SURE YOU UPDATE the OCZ firmware before you start installing all Your software, so that You will get 1.5 gyga read writes per second from ur drive.

You will find a workstation for Your budget on dell outlet for a refurbished machine that will come with Full 3year expandable warranty.

http://www.dell.com/us/dfb/p/precision-desktops?~ck=anav
http://ocz.com/consumer/revodrive-3-x2-pcie-ssd

If You live in europe You can purchase additional hardware from

http://www.scan.co.uk/
they ship in all europe

 

[BenK1010]

The new generation CPUs might be better but they will also be very expensive. And we can't afford 4 of them but that would be nice.

As for overclocking, I saw somewhere that Intel CPUs only support 1333MHz memory speeds but it's not true. See, for instance, http://ark.intel.com/products/64582/. I guess that statement I saw was written a long time ago in terms of CPU tech.

I don't like the idea of buying a refurbished workstation. That's like buying a refurbished race car.