Cheap DIY Render Farm... Ideas?

jjhalodude

Commendable
Jan 7, 2017
1
0
1,510
I do a lot of 3D CAD modeling for a living (Autodesk Inventor mostly). I built my workstation to handle these operations (4790K CPU and low end workstation GPU). Everything runs great... now I have to start rendering images with Autodesk Showcase. As soon as i started rendering, all my CPU cores shot up to 100% usage and rendering was slow.

I need to find a rendering solution. I have a few thousand models to render, ranging from 3 or 4 parts in a model to 3000 parts in a model.

I looked into my options a little bit. My first option would be to build a workstation with (2) high end Xeon CPUs and (1) motherboard, totaling at a $5000 build. This is a bit pricey for me, I'd like to find a cheaper solution.

I have a rough idea but I'm not sure if it is possible. I would like to link (4) 7700k CPUs, (4) motherboards and (4) sticks of 4gb ram on each board.

Can this be done in a (semi-)simple way? I have built PC's before, but never anything to this degree.
 
Solution


jjhalodude,

The simplest and best cost /performance method for high performing CPU rendering is to buy a used dual Xeon workstation and upgrade it by changing: CPU's, RAM, GPU's, and drives. I've done this three times with Dell Precision T5400 and T5500 and am just now finishing an HP z620.

The z620 actually is being setup for both CPU and GPU rendering and analysis / simulation software. Work includes projects of vastly different scale including architecture and industrial design using Solidworks.

I maintain an HP z420 3D modeling system with fewer but faster cores and higher single-thread rate so the z420 and z620 are a complementary pair.

The z620 was purchased fro $270 and had some cosmetic damage with a lower specification:

HP z620 (Original) Xeon E5-1620 4-core @ 3.6 /3.8GHz) / 8GB (1X 8GB DDR3-1333) / AMD Firepro V5900 (2GB) / Seagate Barracuda 750GB + Samsung 500GB + WD 500GB
[ Passmark System Rating= 2408 / CPU= 8361 / 2D= 846 / 3D = 1613 / Mem =1584 / Disk = 574 ] 7.13.16

I researched first series Xeon E5-2600 CPU's and found there were a large number of 8-core server CPU's being cycled out at very low cost: E5-2670 ,2680, and 2690.

Passmark results:

1. E5-2670 (8-core @ 2.6 / 3.3Ghz)
a. 12502____ 18459 (dual) (CPU Mark)
b. 1620 (Single Thread Mark)
c. 115W (Power consumption)

2. E5-2680 (8-core @ 2.7 /3.5GHz)
a. 13410____ 18833 (dual)
b. 1709
c. 130W

3. E5-2690 (8-core @ 2.9 /3.8Ghz)
a. 14438____ 20826 (dual)
b. 1873
c. 135W

Purchased:

E5-2690's: $152 and $154
z620 2nd CPU/Fan /heatsink/RAM riser: $150
64GB (8X 8GB) DDR3-1600 ECC registered : $128
Quadro K2200 4GB (Taken from the Precision T5500)(Value about $350)
Tesla M2090 6GB GPU coprocessor: $86
Internal USB powered fan for the M2090: $18
HP Z Turbo Drive 256GB AHCI (OS /Programs) (This is based on a Samsung SM951 M.2 SSD) : $150
Samsung 850 Evo 250GB: $82 (Active Projects)
Seagate Constellation ES.3 1TB: $60 (Archive /Storage)(The ES.3 has 128MB cache instead of 64MB)
HP new set of all case plastic parts: $56

The total is something in the $1,400-$1,500 range.

Performance results were very good, resulting in the highest of 249 Passmark z620 system rating:

Analysis / Simulation / Rendering:

HP z620 (2012) (Rev 3) 2X Xeon E5-2690 (8-core @ 2.9 / 3.8GHz) / 64GB DDR3-1600 ECC reg) / Quadro K2200 (4GB) + Tesla M2090 (6GB) / HP Z Turbo Drive (256GB) + Samsung 850 Evo 250GB + Seagate Constellation ES.3 (1TB) / Creative Sound Blaster X-Fi Titanium PCIe sound card / 800W / Windows 7 Professional 64-bit > Logitech z313 2.1 speakers > HP 2711x (27" 1980 X 1080)
[ Passmark System Rating= 5675 / CPU= 22625 / 2D= 815 / 3D = 3580 / Mem = 2522 / Disk = 12640 ] 9.25.16

Cinebench scores were very good as well:

[ Cinebench R15: OpenGL= 119.23 fps / CPU = 2209 cb / Single core 130 cb / MP Ratio 16.84x] 10.31.16

The graphics performance marks don't reflect the performance of the K2200 + Tesla M2090 combination which on the Octanebench test produced a score similar to a Quadro M5000 8GB for less than 1/4 the cost.

When shopping, buy a z620 with a bootblock date of 6/13 as those can use the Xeon E5-2600 v2's, which have some of the highest Xeon clock speeds ever. A z620 with a pair of Xeon E5-2687W v2's (8C @ 3.4 /4.0Ghz) (about $1,000 now) and a good GPU and disk setup could do everything: 3D modeling plus CPU and GPU rendering.

You might consider setting up such a system to have a reasonable GPU rendering capability- the idea of "few thousand models to render" might make GPU rendering a deadline saver and make animations / simulations possible. However, for single images, I always use CPU rendering which produces a noticeably better image quality.

Candidates for this kind of system include the Dell Precision T5600, T7600, T7610, and HP z620, z820. When shopping be aware that the HP z620 requires an expensive riser for the 2nd CPU and it may be preferable to buy the z820 which doesn't use one.

I've upgraded six workstations since 2009 and had 100% reliability and each successive one was also quieter nad had better features such as SATAIII controllers and USB3.0

It's possible to find a system with the target CPU (s) already although that may cost more if the CPU is near the top of the range as is the E5-2690. For your use, I'd suggest the E5-2680 (8C@ 2.7/3.5GHz) which are costing about $100-$120 at the moment.

The z620 is not fully tested as software is being loaded just now, but a 3180 X 2140 Vray RT GPU rendering with a single lighting source ran in under four minutes.

If you have 1000's of renderings, for $5,000, it could be possible to build three such systems within the budget, but check your Autodesk licenses. I use Building Suite Ultimate and am allowed to install on two systems provided I am the only user.

The good feature of this method is that the system is all together and only the upgrade CPUs, RAM, GPUs, and drives have to be plugged in The E5-2690, the CPUs cost $150 instead of the original $2,050. These CPUs have a MTBF of 170,000 hours or 19+ years/continuous running, so buying a three year old one means a very strong possibility for several years reliability.

I can't think of better cost/ performance method to have high speed CPU rendering and a fairly easy system configuration. The key is patience and wise shopping.

However, it's possible to build a very competent 16-core / 32 thread CPU rendering system using all new components for a reasonable cost:

BambiBoom Pixel Cannon Rendermaticshowcasealicious iWork TurboSignature Extreme RendalBlast 9800 ®©$$™®£™©™_1.8.17


CPU: 2X Intel Xeon E5-2630 v3 Eight-Core Haswell Processor 2.4 /3.2 GHz 8.0GT/s 20MB LGA 2011-3 CPU, OEM > $1,340 ($670 ea)

Xeon E5-2630 v3

Passmark:

CPU: 18568 (dual)
Single-thread Mark: 1752
Power: 85W

CPU Cooler: 2X CORSAIR Hydro Series H50 120mm Quiet Edition Liquid CPU Cooler - Intel Only (CW-9060006-WW) > $120 ($59.99 ea)

Motherboard: Supermicro X10DAI-O Dual LGA2011/ Intel C612/ DDR4/ SATA3&USB3.0/ A&2GbE/ EATX Server Motherboard> $363

RAM: 32GB (4X 8GB) Samsung DDR4-2133 8GB/1Gx72 ECC CL15 Server Memory > $228 ($57 each)

GPU: PNY Quadro M2000 VCQM2000-PB 4GB 128-bit GDDR5 PCI Express 3.0 x16 Workstation Video Card >$427

Drive 1: Samsung 950 PRO Series 512GB M.2 PCIe 3.0 x4 Solid State Drive, Retail (V-NAND) > $318 (OS and Programs)

M.2 to PCIe X4 adapter: Lycom DT-120 M.2 PCIe to PCIe 3.0 x4 Adapter >$22.85

Drive 3: Seagate Constellation ES.3 2TB 7200 RPM 128MB Cache SATA Internal Hard Drive > $127.70 (Storage)

Power Supply : Seasonic SS-1050XM2 ATX 1050 Power Supply > $197.00

Case: Lian-Li Case Full Tower Chassis Aluminum USB3.0 Black Retail PC-A76 > $242.02

OS: Operating System: Microsoft Windows 7 Professional SP1 64-bit English (1-Pack), OEM > $139.

_______________________________________

TOTAL= $3,467

Here's a solution with a better cost/performance ratio plus an easier assembly /configuration than starting with all separate components:

BambiBoom PixelCannon Showcaseagrapharific iWork TurboSignature Extreme ModelBlast 9800 ®©$$™®£™©™_9.13.16

This concept is based on a Supermicro Superworkstation. This provides a case, dual LGA2011 motherboard, CPU coolers, and power supply such that the user need only plug in the CPU's, RAM, GPU's, and drives. This greatly simplifies the hardware decisions and makes configuration very fast. the cooling is designed for server-level use and these systems are rated to be very quiet.

The cost/performance key to this idea is employing used CPU's. By using depreciated CPU's, a much higher specification is possible and funds are released for other purposes.

Case /Motherboard / CPU coolers / 900W Power supply : Supermicro SuperWorkstation SYS-7037A-I Dual Socket LGA2011 Xeon 900W Mid-Tower Workstation Barebone System (Black) > $740

http://www.supermicro.com/products/system/tower/7037/SYS-7037A-i.cfm
http://www.superbiiz.com/detail.php?name=SY-737AI

CPU: 2X Intel Xeon E5-2690 8C@ 2.9/ 3.8GHz, 20MB LGA 2011 CPU, 135W > $400 (used about $200 each)

http://ark.intel.com/products/76161/Intel-Xeon-Processor-E5-2687W-v2-25M-Cache-3_40-GHz

Passmark:

E5-2690 (8-core @ 2.9 /3.8Ghz)
a. 20826 (dual)
b. 1873
c. 135W

Memory: 64GB (8x 8GB) Samsung DDR3-1866 8GB/512Mx8 ECC/REG CL13 Samsung Chip Server Memory > $488 ($61ea.)

http://www.superbiiz.com/detail.php?name=D3-18R8GS2

GPU 1: PNY Quadro M2000 VCQM2000-PB 4GB 128-bit GDDR5 PCI Express 3.0 x16 Workstation Video Card > $426.

http://www.newegg.com/Product/Product.aspx?Item=N82E16814133624&cm_re=Quadro_m2000-_-14-133-624-_-Product

[Disk 1: Samsung SSD 950 PRO 512GB NVMe M.2 Series PCI-Express 3.0 Internal MZ-V5P512BW with 2.5" SSD Plastic case (SSD) > $358

http://www.newegg.com/Product/Product.aspx?Item=9SIA12K4871049&cm_re=samsung_950_pro-_-9SIA12K4871049-_-Product

PCIe M.2 Adapter: Lycom DT-120 M.2 PCIe to PCIe 3.0 x4 Adapter (Support M.2 PCIe 2280, 2260, 2242) > $22

https://www.amazon.com/Lycom-DT-120-PCIe-Adapter-Support/dp/B00MYCQP38/ref=pd_lpo_147_lp_t_4?ie=UTF8&psc=1&refRID=BXP759K2XTHDRYCWFRY4

Disks 2, 3: Seagate Constellation ES.3 ST3000NM0023 3TB 7200RPM SAS3/SAS 6.0 GB/s 128MB Enterprise Hard Drive (3.5 inch)> $183 (Files, Backup, System Image)

http://www.superbiiz.com/detail.php?name=ST3000NM23

Optical Disk: ASUS 24X DVD Burner - Bulk 24X DVD+R 8X DVD+RW 12X DVD+R DL 24X DVD-R 6X DVD-RW 16X DVD-ROM 48X CD-R 32X CD-RW 48X CD-ROM Black SATA Model DRW-24B1ST/BLK/B/AS - OEM > $20

http://www.newegg.com/Product/Product.aspx?Item=N82E16827135204

Operating System: Microsoft Windows 7 Professional SP1 64-bit English (1-Pack), OEM > $139.

http://www.superbiiz.com/detail.php?name=MSFQC08289
_______________________________________

TOTAL = about $2,776

Note that the performance of the E5-2690 is better than the E5-2630 v3 at less then 1/3 the cost.

One question about the above system is to have a good cooling solution for the Samsung 950 Pro

In my view, the Supermicro Superworkstation is the best solution in terms of overall time / effort / cost /performance and only the CPU's are used, but for the fastest processing density- getting through thousands of renders- upgrade a pair of used workstations.

Cheers,

BambiBoom

CAD / 3D Modeling / Graphic Design:

HP z420 (2015) (Rev 3) > Xeon E5-1660 v2 (6-core @ 3.7 / 4.0GHz) / 32GB DDR3 -1866 ECC RAM / Quadro K4200 (4GB) / Samsung SM951 M.2 256GB AHCI + Intel 730 480GB (9SSDSC2BP480G4R5) + Western Digital Black WD1003FZEX 1TB> M-Audio 192 sound card > 600W PSU> > Windows 7 Professional 64-bit > Logitech z2300 2.1 speakers > 2X Dell Ultrasharp U2715H (2560 X 1440)
[ Passmark Rating = 5581 > CPU= 14046 / 2D= 838 / 3D= 4694 / Mem= 2777 / Disk= 11559] [6.12.16]

Previous rendering system:

Purchased for $171:

Dell Precision T5500 (2011) (Original): Xeon E5620 quad core @ 2.4 / 2.6 GHz > 6GB DDR3 ECC Reg 1333 > Quadro FX 580 (512MB) > Dell PERC 6/i SAS /SATA controller > Seagate Cheetah 15K 146GB and 300GB > Windows 7 Professional 64-bit
[ Passmark system rating = 1479 / CPU = 4067 / 2D= 520 / 3D= 311 / Mem= 1473 / Disk= 1208]

Result:


Dell Precision T5500 (2011) (Revised) > 2X Xeon X5680 (6-core @ 3.33 / 3.6GHz), 48GB DDR3 1333 ECC Reg. > Quadro K2200 (4GB ) > PERC H310 / Samsung 840 250GB / WD RE4 Enterprise 1TB > M-Audio 192 sound card > Logitech z313 > 875W PSU > Windows 7 Professional 64> HP 2711x (27", 1920 X 1080)
[ Passmark system rating = 3844 > CPU = 15047 / 2D= 662 / 3D= 3550 / Mem= 1785 / Disk= 2649] (12.30.15)










 
Solution