Dual Xeon Build - Questions and looking for advice/suggestions

kuakman

Prominent
Oct 21, 2017
22
0
510
Hello Everyone,

My first post here at tom's hardware. I've been looking for a forum like this one to ask a few questions (and some advice) by experts out there in the community. I'd appreciate your comments because that will help me to make the right decisions since I'm kinda of a noob regarding building my own workstation.

So in simple words, I'm planning to invest money to get a nice and solid workstation for mostly doing 3d animation / rendering (Not for Gaming). Software like, Blender, 3dstudiomax, Maya, Zbrush and Photoshop. And maybe, Substance Designer/Painter. These are just examples to give you guys an idea.

Currently the budget I was planning to invest is around 5k, if it's more than that, I will be willing to pay based on if that really justifies the need.

So I did some research out there and seems like dual xeons will be a perfect fit to what I'm looking for.
Checking the prices out there, here is what I came up with:


  • - CPU: 2 X Intel - Xeon E5-2630 V4 2.2GHz 10-Core
    - Motherboard: Supermicro - MBD-X10DAX EATX Dual-CPU LGA2011-3
    - Memory: 4 X Kingston - ValueRAM 16GB Registered DDR4-2133
    - Cooling: 2 X Corsair - H80i v2 70.7 CFM Liquid CPU Cooler (One per each CPU)
    - GPU: MSI - GeForce GTX 1080 Ti 11GB GAMING X
    - CHASIS: Thermaltake - Suppressor F51 (Windowed) ATX (based on the specs supports EATX)
    - Power Supply: Corsair - HX Platinum 1000W 80+ Platinum Certified Fully-Modular ATX
    - Storage:
    - 1 X Western Digital - Blue 2TB 3.5" 5400RPM Internal Hard Drive
    - 1 X Intel - 545s 512GB 2.5" Solid State Drive
Starting with the xeons, I tought that a Xeon 2630 v4 relation power vs cost will be a good match for my budget, but I'm open for alternatives.

One of the main questions I have is the case... considering that I'm getting the Supermicro which seems to be a "server" motherboard based on dimensions, I'm gonna need a EATX based chasis, but I also need to consider that heatsinks + the corsair H80i water coolers and everything will fit in the case.
So my first bet was the Thermaltake - Suppresor F51 which seems that it supports EATX (12' x 13') motherboards. (Good price) but I have some doubts about it.
Do you guys have any suggestions or have you ever have experience to fit all these components on case with dimensions that will support all of this?

Regarding the cooling system, (because I've never own a xeon processor before) and reading articles about it, it seems that water cooling based are not technically necessary, I read that some people can use the classic air cooling fans. what do you guys think? I have seen other models from corsair like the H115i which seems to be better, but I'm concern about the size or compatibility with the processors.
But at the same time, I wouldn't mind to spend extra money to get an great cooling water-based system to keep the temperatures down as much as possible, and expect to get better performance and throughput from the CPUs.

Another question that pops up in my head is Memory... I want to start with 64 GBs (4 modules DDR4 2133) which the super micro seems to support based on the specs. The other aspect is that the memory modules should support ECC. I heard that some people have mentioned in some forums that it's highly recommended to distribute evenly equitable across the CPUs. This is: 32GB per processor, otherwise you will hit some performance degradation by the xeons. Not sure if this is part of a myth or not. Anyways, 64 GBs just to start and keep expanding from there if I need to. Any suggestions?

And the last question is about the GPU... I know some people will say "hmmm - a 11GB 1080TI may be too much for a non-gaming rig...", I can probably go with something cheaper like a GeForce 970 and save a few bucks and use it for a better chasis. Suggestions about this?
Also, considering I already have 2 Monitors + a Wacom Cintiq 22" to plug into this new rig.

Regardless to say while doing research, I felt in love with the In-Win D-frame 2.0, but spending 1.5k in a fancy chasis seems like too much. But I thought about it. Not sure if the components will fit on this one though.

Alright, Thanks to the community!
Sincerely, I will appreciate your comments.
Thank you!
 
Solution
I never said air cooling is a better than water cooling. One of the reasons, enthusiast (ocing), wouldn't make sense if water was worse. I choose my words carefully and there is no interpreting or reading in between the lines needed. If you want to re-read what I typed than you'll see what I said is logical. I know I may not go in depth in everything but it's really off topic if you don't need it plus there is a ton of info on these subjects that books are literally written. But I have no issue diving deeper if that is what's wanted. We're here to help in anything hardware/software related.

240m and 280mm refers rad sizes. I said "240 or 280mm rads are the only aios even worth looking at." I specifically say rads and also aios. Maybe...

KittyFish62

Respectable
Dec 13, 2016
567
0
2,160
It might be better to go for a threadripper or an i9, as those have more cores and threadripper is much cheaper than a xeon or an i9, however you'll need to have really really good cooling for both threadripper and an i9. I'd say go for the 1080ti, although I'm not very experienced with servers so I might not be right.
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Thanks for your comment, yea. I was also looking for a single physical processor like the new I9 that Intel released recently and that will match my budget, like the 7920X (https://www.newegg.com/Product/Product.aspx?item=N82E16819117811)
But doing some simple math, that processor has 12 cores (24 total threads with hyper threading activated), but compared to the dual xeon, you will have 10 cores (20 threads) x 2 = 40 threads total.

And this is what I'm not sure about, whenever you start rendering a complex 3d scene that needs to perform a lot of calculations like reflections, shadows, light casting and so on... the 40 threads may perform better than the 24 from a single physical cpu, general speaking. the question here is: are there any other factors, like the cpu clocking speed 2.2 GHZ vs 2.9 GHz that will make the difference in terms of performance.

Another thing to point at, when you use let say mental ray as a render engine, seems like you can decide how you split the "buckets" (areas of a image to render), and seems like this render engine in particular assigns a single thread to a portion of the image being renderered. Here a screenshot if you are not familiar with it (https://bw-1651cf0d2f737d7adeab84d339dbabd3-forumpro.s3.amazonaws.com/file_453014.JPG)

Anyways, thanks for you comment, I will look at the threadripper...
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Weird I double checked on newegg.com and the price of a E5-2630 v4 is $669 each + some taxes I assume.
Here the link: https://www.newegg.com/Product/Product.aspx?Item=N82E16819117630&nm_mc=KNC-GoogleKWLess&cm_mmc=KNC-GoogleKWLess-_-DSA-_-CategoryPages-_-NA&gclid=EAIaIQobChMIkdvG5f6C1wIVB2x-Ch2iJAwmEAAYAiAAEgJdk_D_BwE&gclsrc=aw.ds
 

mwoolfso

Prominent
Oct 21, 2017
1
0
510
First off, my initial thoughts....I am a fan of SuperMicro Server solutions, just not in this instance when requirements are 3D animation and rendering

- 3d rendering is more about the GPU and RAM solution and less about the CPU. Storage would be either above or below CPU based on need
- a single CPU like a top-of-the-line i7 would be better IMHO. With i7 you get the ability to overclock and can extend the life of the rig over time. ECC memory is slower than non-ECC memory due to the extra maintenance cycles

My recommendation would be a SuperMicro C9X299-PGF with an i7; the board supports i7/9 and you can always upgrade to an i9 in 2 years time without the cost premiums you would experience today if you started with an i9. The board supports non-ECC DDR4 RAM up to 128GB so same approach, start with a good estimate and you have the ability to upgrade should the need arise.

In general you won't need liquid cooling unless you want visual bling. 4-pin fans, the right case and motherboard in the right room with good cooling will take care of the parts for you easily enough.

I'm surprised with your storage approach. First off, motherboard SATA slots are notoriously SLOW. You don't notice because SSD drives mask the issues and if I/O is a significant need then you go for an AVAGO SATA Host Bus Adapter which gives you the ability to run the drives in a RAID or non-RAID configuration depending on the firmware you use. The OS can boot off the motherboard ports or the HBA but any data disks would run off the HBA. The folks on the web call it RAID-mode vs. IT-mode. If you are using Windows then RAID mode is my recommendation. I've done both and yeah there is a difference in performance for long-running processes. If you need 2TB of storage look for multiple drives in a RAID configuration. I gravitate towards notebook drives in a 4,6,8,12 bay enclosure - some drives can be established in a hot-spare configuration so if one drive goes you are not down. Replacing drives while the system is on is relatively easy. I would recommend (4) Seagate 2TB Mobile Drives w/ 128MB cache, put them in a RAID5 or RAID10 configuration and you are set. I would gravitate towards the RAID5 with 1 drive being a hot spare. You would then have 4TB of usable space. Some folks say RAID5 is slow and while it MAY be slow... the dedicated processor on the HBA mitigates that risk completely, and if you get an Avago HBA adapter with 512MB or 1GB of cache with battery, again far less risk.

Lastly, get a battery backup UPS and connect it to the main system via USB. This way you can get even more protection for the rig.
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Thanks for your reply! I really appreciate it.

Here is my answer from my humble opinion, again I'm kinda of a noob but I had some experience before using software like 3dstudiomax and maya, along others. And this is because GPU vs CPU usages regarding the term "rendering" can be quite ambiguous sometimes. Not trying to get into an argument here, but let me put some words in here to highlight what I know, based on my experience and then you can vet my theory if you think is pertinent.

When trying to understand how the GPU is being used by different applications, that being said, this can be a game or simply an application, I always thought the GPU is in charge of the "volatile" information that the application needs to send to the GPU, that will end up being displayed on the screen at the moment an application requests it. After that, the information is gone, immediately "garbage collected" so to speak.

So if we put this concept, a game will use intensively a lot of GPU, not question about it. So, imagine a first person shooter game, drawing 3d geometry, calculating reflections, textures... all in real time, as soon as you move your character around inside a world scene. I'd say that the CPU is important also, but the cpu is constantly sending information to the GPU. So let's say we can call this "live rendering" process right? CPU and GPU are highly demanded by the game.

Now let's go back to the 3d software packages context, you have usually viewports (workspaces) and you have some sort of real time updates in those viewports base on what you do. You can be either modeling a character, texturing (generting UV Maps, baking normal maps and so on).
On this instance, you could also say that the GPU is being used so far. But when the time comes to "render" (again same concept), it seems to me, what I perceived in the past because I've been monitoring the CPU usages when this happens: the CPUs went up to 100% because the software is demanding a lot of "calculations" that are not necessary requests for displaying objects on the screen "constantly" as I mentioned previously with the game example but rather, generating a final "image" which should be the result of you have done in those viewports. Again, the GPU will be used here, but I think... not as much as the CPU.
Not sure if you are following me...

That's why I must say I kinda of disagreed, but again you can vet my theory, because this is an explanation of what I empirically perceived in the past working on the 3d world, using single physical CPUs.

Regarding the storage, you might be on the spot.
This is what I had in mind when I thought of storage options:

SSD (and I'm not sure if this is true) they supposed to be "faster" so when I want to open applications, boot my system and open 3d software application for working with 3d scenes, loading textures and so on, those requests to open files, load should be "quick" so I thought: "512GB for apps and using them for projects must suffice".
Now whenever I complete a 3d project and I'm not longer using those projects files, I will be backing them up (to have a backup copy) into the SATA Western Digital (2 TB bigger of course and if it's slower, it doesn't actually matter because if for the storage).
Yea the UPS is a good idea. Again this is not for a server, I'd say it's a dedicated personal workstation rig for 3d only. So I don't think RAID will matter, RAID as my understanding goes, is when you need to deal with massive amount of data to be store and move it quite often from one place to the other (Backup restoration, Databases and so on).

are these good assumptions? or am I wrong about something?

Thank you again for your response!
 


Some rendering engines do all of their final rendering on the CPU, while others, like Octane, do it exclusively on the GPU. If the software you use depends entirely on CPU performance for final renders, then going with lots of CPU cores might make sense. If it uses the GPU though, then going with maximum graphics card performance could be better. It might be worth checking forums or support for the particular software you use to find what hardware might run it best.
 
Cores and ghz aren't created equal and can't be compared over different architectures. For simplicity, just look at real world benchmarks to gauge cpu performance.

You seem to have a good idea on hardware usage which is uncommon for most 3d artists actually. But there is a difference between a cpu based renderer like mental ray vs a gpu renderer like iray or cycles (cycles can be selected as cpu or gpu). Most 3d apps still don't have gpu renderers built in like maya but many use 3rd party renderers. Render times are largely dependent on what's being rendered and could be faster on cpu or gpu. It's one of the reasons why places like pixar or dreamworks use a combo of each.

But rendering is never gpu and ram. When it's a gpu renderer, the scene is loaded to vram. When it's a cpu renderer then gpu is irrelevant. Not enough vram for a gpu renderer and it crashes, the same as not enough ram for a cpu renderer. Modeling and animation is dependent on viewport performance which would be gpu. This is probably where the gpu and ram came from as you need enough ram for working files but don't mix rendering needs with other work needs. Baking is not viewport work, it's actually like rendering as it's a separate function and can similarly be cpu or gpu based.

Going with lower end to upgrade later costs more in the long run. If the needs/benefits are there now, there is no reason why someone should buy a cheaper build. Before I can really suggest components, I'd want to know what specs you work on now, what type of work do you even do, and what renderers do you use? In any case, you'd want some experience with gpu rendering if you haven't used it to see if they would suite your needs before spending the money on a new build that could have different needs. In many cases gpu rendering is faster but rendering is complex, 3d work is very diverse and I'm not saying it's always faster. It does not fit everyone's needs as some features will not work with it.

There are some definite changes as mentioned aio coolers are just a waste of money. Your storage needs some improvements. You don't want to spend more on the case (just for looks) vs cpus. That shouldn't have been needed to be said.
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Yess, I recently installed Blender (of course because is open source while autodesk's software is nice but quite expensive) I noticed that, some render engines have incorporated the feature that allow the use of GPU to render and I thought: - "What is that?? seriously? That's new... I didn't know about that" haha.
While I was gone for a while from doing 3d work (like 5 years ago), I used to use mental ray a lot and in previous versions, MR required a lot of CPU to get the final render for a complex scene.

So what you mentioned is very helpful, and that might be something to consider while picking the right components in relation with what software and render engine I will be using. Good catch!

So the "gaming gpu" 1080 TI might not be a bad choice after all. I will end up taking advantage of those render engines.

Thanks!
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Got it!
So to recap:
- For xeons you still recomend water-based cooling system. (thumbs up)!
- Regarding the case, I guess you are right and I don't mind something that it doesn't look as fancy as the D-frame 2 by In Win haha :)D) . I guess a decent case for not more than $200 should be ok, as long as everything fits in there and is cheaper, the better.
- Regarding the storage, what would you suggest? Do you think that just with the SATA WD 2TB 5400 will suffice? or am I on the wrong spot with what I mentioned here:

SSD (and I'm not sure if this is true) they supposed to be "faster" so when I want to open applications, boot my system and open 3d software application for working with 3d scenes, loading textures and so on, those requests to open files, load should be "quick" so I thought: "512GB for apps and using them for projects must suffice".
Now whenever I complete a 3d project and I'm not longer using those projects files, I will be backing them up (to have a backup copy) into the SATA Western Digital (2 TB bigger of course and if it's slower, it doesn't actually matter because if for the storage).

Thanks Much!
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Forgot to answer the main question to you reply:

Just to give you an example of what I'm planning to work on and (learn more):
Mostly, Realistic Character/Creature Design (Zbrush), Particle Simulation (using Blender)
Here a few examples I found out there:
- Creature Sculpting: https://www.youtube.com/watch?v=QqbTIIPZbz8
- Particle simulation: https://www.youtube.com/watch?v=x8Fo2slT2WA

These might be extreme examples, but that's what I'm aiming to.
So I guess the 1080 TI might still be an option. I agreed, 3d software is complex... there are a lot of render engines out there with different capabilities. So I would say based on the CPU - GPU discussion, I would try to have a "balanced" build, this is: trying to take advantages of both of CPU or GPU "rendering" whenever possible. CGI artists use different pieces of software so it will be hard to answer that question (if it's a CPU or GPU demand). I'm looking forward to build a base "semi professional" rig to start with and the possibility to make the hardware scale whenever I need to.

 

mega maniac

Distinguished
Aug 5, 2010
37
0
18,530
*if* cores are the most important consideration, (and thats a big if, some software does not scale well beyond a certain amount of cores) then have a look on ebay for the Xeon 2696v4 - the same as the 2699v4 but a fraction of the price as removed from OEM workstations. Also make sure you get a matching pair.
 

samer.forums

Notable
BANNED
Sep 30, 2017
662
0
1,160
Why are you looking at the old Xeon system/CPU ?

look for the new ones . socket 3647

Multi CPU

https://www.supermicro.com.tw/products/motherboard/Xeon1333/#3647

Single CPU

https://www.supermicro.com.tw/products/motherboard/Xeon1333/#3647

dont waste your money on the old platform.

New scalable xeons 2017

https://www.intel.com/content/www/us/en/products/processors/xeon/scalable.html

New workstation Xeons 2017 (single chip)

https://www.intel.com/content/www/us/en/products/processors/xeon/w-processors.html
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Thank you for your reply!

I was also looking at ebay for more powerful processors you can get for a cheaper price, like the 2696v4 you mentioned where the commercial prices as "new" are around $1.8k-2k. But I'm kinda of hesitating to get a cheap "used" one (refurbished or in some cases I read some resellers, are offering them waaay to cheap because they got them from the manufacturer but they were not well tested). In other words, not sure if I should take my chances on those, I'm tempted to go for it but I also read some good and bad experiences from users doing this.
Anyways, I recognize that this may be a way to take huge advantage if you get a pre-owned one for half of the price as new and better processor with more cores.

I'd like to say this is still an alternative because of the characteristics of Xeons makes me think it could be safe to buy pre-owned for the simple fact that xeons cannot be overclocked or "beaten" up so to speak.

Finally, I'm trying to make a list of all the options I have based on suggestions/opinions from people posting here while I keep doing research. This answer is in my list though and I appreciate it ;)

 

kuakman

Prominent
Oct 21, 2017
22
0
510


Yup, you are on the spot! The Scalable Xeons with the new socket architecture look promising.
Let me take some time to read those specs and check the prices. I will get back to you. Thank you!

I will be refining my original list of components that will match my budget with those new cpus.
Also as @mega maniac mentioned, I will check prices on ebay as well and compare.
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Gotcha! No worries... I was chatting here with a few folks that they also provided excellent input. So I'm still planning to get the 1080 TI though :). Right now I'm researching CPUs, gathering all the feedback from all the users and apply more modifications to my list of components.

Thank you so much!
 

kuakman

Prominent
Oct 21, 2017
22
0
510
Ok here an update on the component list researching the new Xeon Architecture Socket 3647:

ySL-_PGbfbugzf58zv0tqXNdC_EmW_dLTDB75KZ01m0


On Actually I forgot to update the RAM. That mobo seems to support 2666MHz clocking speed. But those are the numbers I got.
A third alternative is to buy a dual socket mobo, but buy a single cpu (more powerful) like the Xeon Gold 5120 - 14 cores (28 Threads) on 2.2 GHZ at $1585.99, and work with a single cpu for now. And later, once I save more money I can keep scaling and add a second one.
Not sure if these mobos support the usage of a single cpu and leave a socket empty. Any suggestions?
 
I do not recommend water cooling. I said it was a waste of money. I'm not sure how that was interpreted incorrectly. Wc is for hobbyist (looks) and enthusiast/extremist (ocing). In some cases the aios are good for small form factors otherwise it's people who have extra money that get them. Although a custom loop is much better, the bigger 240 or 280mm rads are the only aios even worth looking at. Air coolers offer the same cooling performance at a lower price, are often quieter, you don't risk leaks that will short and kill your pc, or the pump failing (somewhat common on aios) and you overheat. Wc is the opposite of what you want in a workstation.

Used cpus can be a good value depending on the price you can get them. There is no such thing as a cpu being "well tested." Other than worries about it being badly oced then it'll last for more than a decade and past it's usefulness. Tray cpus are not a lot cheaper and is not the reason they are way cheaper. Businesses typically have partnerships with pc manufacturers and update every couple of years. This leaves large inventories of pcs on the cheap whenever they swap out pcs. They're typically only lightly used and people rarely have issues. It's cheaper to buy a used pc whole and upgrade certain parts vs buying used parts separately. If looking at new prices, you would get the newest cpus for the best value as the price brackets don't change but the cpu specs improve.
 

kuakman

Prominent
Oct 21, 2017
22
0
510


Ah, my bad. I miss-interpreted this statement:
There are some definite changes as mentioned aio coolers are just a waste of money [...]
 

kuakman

Prominent
Oct 21, 2017
22
0
510


On a side note, I must say it's debatable.

Agreed that hobbyist may prefer wc for esthetics reasons if they have the money and they want to pay for it. In the other hand, I consider that "Air" is more of an "insulator" rather than it is a conductor of heat. While water cooling seem more risky (because of possible leaks as you mentioned), liquid and solid materials generally speaking, are better conductors of heat to the sole goal of dissipating it. So, water coolers usually are more efficient at that.

Of course I'm NOT saying that we cannot find out there a good Air Cooler that can be as effective as Water coolers to dissipate the same amount of heat (at a better price), but I must say wc systems in general are more efficient because of the reason I mentioned before.

Again, regarding the form factor which seems to be this case for the Xeons cpu proposed as part of the build (and this was one of my main questions, I appreciate your suggestion)... you are right, it might not be worth the investment money on water coolers.

Thanks!
 

mega maniac

Distinguished
Aug 5, 2010
37
0
18,530
Yea, I wouldn't water cool Xeons, you wont even need top end air cooling. So long as your cooler keeps the CPU cooled adequately at its peak operating frequency your golden as overclocking the CPU isn't possible.
 


Except that liquid coolers still ultimately need to dissipate their heat using air. The water carries heat away from the CPU and to the radiator, but then then you're still relying on fans to transfer that heat from the radiator to the air. Any heat not transferred into the air remains in the water getting pumped back to the CPU. So really, it's still technically air cooling.

The water's main purpose is to allow the radiator to be located somewhere away from the CPU. From there, it can either draw cooler air in from outside the case and dump the warmed air inside, or draw air from inside the case, and push the warmed air outside, though really, a tower cooler setup with good airflow could perform similar in many cases. It is possible for some liquid coolers to incorporate larger three-fan radiators though, which can potentially cool better than a tower cooler due to having more surface area and fans.

One other thing the water in a liquid cooler can do is act as a buffer for the heat. The CPU temperature will tend to rise more slowly, since it has to warm not only the radiator, but also the water. So for short bursts of activity, you'll likely see lower temperatures. However, on sustained loads, eventually the water temperature will rise, and once that happens, the temperatures will tend to be similar to a tower cooler with a comparably-sized radiator and fans. And once the load is removed, it will take longer for a liquid cooler to cool back down compared to a tower cooler, since it needs to cool all that warmed water as well, and can only do so as fast as the radiator can cool it. The size of this buffer can be increased by incorporating a reservoir to hold additional water, but most all-in-ones don't contain a separate reservoir, or a particularly large amount of water.
 

TRENDING THREADS