Build for Multiple VMs

Toggle sidebar Toggle sidebar
S

SSri

Distinguished
Mar 11, 2010
503
1
19,015
Hi all,

I need a little help in deciding hardware requirements for a new build (s).

My requirements are as follows, I'm sorry, there are too many requirements here. It is not easy to fit all in one! :)

1) PfSense-VPN as one VM
2) Suricata/Packet Inspection as second VM. I will leverage my cisco managed switch to manage / route all vlan traffic and avoid them routing through the PfSense router.

3) Plex to stream at least 1080p (2) and ideally one additional 4k.
4) Nas / File Sever
5) Homelab running 4 vms as one cluster. Ideally. I may probably need a dual xeon only for this to run the Hadoop/Spark cluster. This is an overkill as I can run them on a single VM. Either it has got to be a single VM or 4 VMs at least.
6) I have not considered a back up solution for any of them. This is critical. Perhaps, I will grab a couple of High capacity external hard disks and keep them offsite.

My bad. I have neither thought of a budget to address requirements (3) -(5) nor set £££ aside. I thought, may be I will pick up a couple of old xeons off ebay for these purposes. The TDP of two xeons scare a hell out of me. :)

I have settled on the following spec for (1) and (2) on a single machine.

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i7-7700K 4.2GHz Quad-Core Processor (£299.94 @ Aria PC)
CPU Cooler: NZXT - Kraken X62 Liquid CPU Cooler (£149.99 @ Novatech)
Motherboard: ASRock - Z270M Pro4 Micro ATX LGA1151 Motherboard (£105.91 @ BT Shop)
Memory: Corsair - Vengeance LPX 16GB (2 x 8GB) DDR4-3000 Memory (£123.56 @ More Computers)
Storage: ADATA - Ultimate SU800 128GB 2.5" Solid State Drive (£52.17 @ Ebuyer)
Storage: ADATA - Ultimate SU800 128GB 2.5" Solid State Drive (£52.17 @ Ebuyer)
Case: Corsair - Air 540 ATX Mid Tower Case (£124.05 @ Ebuyer)
Power Supply: EVGA - SuperNOVA G3 (EU) 550W 80+ Gold Certified Fully-Modular ATX Power Supply (£78.24 @ Aria PC)
Total: £986.03
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-05-29 22:04 BST+0100

I would love to sport Ryzen 1700 and overclock them. But, the PSOD on ESXi is a problem. I see a little point in installing ESXi disabling SMT.

BTW, I want to set up Raid 1, partition the Raid 1 array into two. A small slice for ESXi and the rest for 2 VMs.

Questions:

(a) Which is better please for WAN-Switch (Lan)?

intel i350 t4 (£41) or i540-t2 (~£100) for (1) and (2). I am on a 350/20 cable service.

http://www.ebay.co.uk/itm/Genuine-Intel-Quad-Port-Server-Ethernet-Adapter-I350-T4-PCI-Express-I350T4BLK-/302320524446? hash=item4663b51c9e:g😛1gAAOSwX61ZHVV8

or

http://www.ebay.co.uk/itm/Intel-X540-T2-10G-Dual-RJ45-Ports-PCI-Express-Ethernet-Converged-Network-Adapter-/131945268901

(b) Should I run (1) to (5) on a single machine and spin off as separate VMs or am I better off building two systems one each for (1)-(2) and (3) to (5) please?

Sorry. It is a long post. Thanks for reading.
 
Solution
USAFRet
1. Your pfSense should be run in its own physical box.
You can do it as a VM, but that is a real pain, and you'd need multiple LAN ports.
An i3 would suffice for that.

2. For your main VM box....more RAM.
Each VM takes a chunk of your RAM.
This is my current i7-4790k, 32GB RAM, running 4 VM's.
ZDia7Gh.jpg

Win 10 Pro, Server2016, LinuxMint, Ubuntu, and the host Win 10 Pro.


Backups? Buy a NAS box.
I have a 4 bay Qnap TS-453A. 4 x 3TB drives.
Sort by date Sort by votes
1. Your pfSense should be run in its own physical box.
You can do it as a VM, but that is a real pain, and you'd need multiple LAN ports.
An i3 would suffice for that.

2. For your main VM box....more RAM.
Each VM takes a chunk of your RAM.
This is my current i7-4790k, 32GB RAM, running 4 VM's.
ZDia7Gh.jpg

Win 10 Pro, Server2016, LinuxMint, Ubuntu, and the host Win 10 Pro.


Backups? Buy a NAS box.
I have a 4 bay Qnap TS-453A. 4 x 3TB drives.
 
Upvote 0 Downvote
Solution
@USAFRet:

Thank you. That's basically, running my first requirement (PfSense and VPN) completely on its own. I want to maximise my VPN throughput as much as possible; VPN throughput maximisation is basically as a future proof, which I know is stupid as technology becomes quickly outdated. OpenVPN being single threaded, a Kabylake cpu with the highest clock speed should do the trick.

Do you think a single (second) build would be enough to address other requirements (2) -- Suricata/Packet Inspection -- to (5) Homelab ? I reckon I would need a dual CPU (xeon, perhaps).

I assume there are no challenges running Suricata on a small VM than running it jointly with PfSense. I do not want to run that with PfSense. Instead, I plan to set up port mirroring in my cisco managed switch and use the small VM to inspect packets.

Thanks,
Sundar
 
Upvote 0 Downvote


Thank you. That's good to know.

I do want to OC i3 7350K/ . But, not sure if Kranken X62's heavy price premium vs Nocuta NH-D15 is worth it. I'm not sure if I should delid it. I live in the UK. The CPU load may kick in when I use VPN.

This is my revised spec for the PfSense box spec. I plan to do Raid 1. I would appreciate thoughts and comments from this forum members.

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i3-7350K 4.2GHz Dual-Core Processor (£147.80 @ Alza)
CPU Cooler: Noctua - NH-D15 82.5 CFM CPU Cooler (£75.49 @ CCL Computers)
Motherboard: ASRock - Z270M Pro4 Micro ATX LGA1151 Motherboard (£105.91 @ BT Shop)
Memory: Corsair - Vengeance LPX 8GB (2 x 4GB) DDR4-3000 Memory (£63.60 @ Aria PC)
Storage: Intel - 320 Series 40GB 2.5" Solid State Drive (£35.99 @ Novatech)
Storage: Intel - 320 Series 40GB 2.5" Solid State Drive (£35.99 @ Novatech)
Case: Corsair - Air 540 ATX Mid Tower Case (£124.05 @ Ebuyer)
Power Supply: SeaSonic - 520W 80+ Bronze Certified Fully-Modular ATX Power Supply (£68.80 @ Alza)
Total: £657.63
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-05-30 09:31 BST+0100

Intel i350 t4 - £41.

http://www.ebay.co.uk/itm/Genuine-Intel-Quad-Port-Server-Ethernet-Adapter-I350-T4-PCI-Express-I350T4BLK-/302320524446?
'
 
Upvote 0 Downvote


I have no plans of OC at this juncture. This is something I may consider to maximise VPN throughput if I hit the limit. Otherwise, there is no need to get the K series.


just for redundancy.


I thought so. I will look around for a new case, post a final spec and close this out this week end. I will be travelling from this evening through the week on business.


Fair enough!

Thank you sir!
 
Upvote 0 Downvote
Revised. I have dropped Raid 1. I am not sure it is a good idea to put it on SSD. I will, in any case, take a back up. I can live with it for a couple of days if the disk fails.

My revised parts list. I am still not sure if I should replace air with water cooler (corsair h80i v2 or h100i v2).

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i3-7350K 4.2GHz Dual-Core Processor (£147.80 @ Alza)
CPU Cooler: Noctua - NH-D15 82.5 CFM CPU Cooler (£75.49 @ CCL Computers)
Motherboard: ASRock - Z270M Pro4 Micro ATX LGA1151 Motherboard (£105.91 @ BT Shop)
Memory: Corsair - Vengeance LPX 8GB (2 x 4GB) DDR4-3000 Memory (£63.60 @ Aria PC)
Storage: Intel - 320 Series 40GB 2.5" Solid State Drive (£35.99 @ Novatech)
Case: Thermaltake - Core V21 MicroATX Mini Tower Case (£53.99 @ Amazon UK)
Power Supply: Silverstone - Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (£81.60 @ Kustom PCs)
Total: £564.38
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-05-30 13:06 BST+0100

Another 40 quid for intel i350 t4.

http://www.ebay.co.uk/itm/Genuine-Intel-Quad-Port-Server-Ethernet-Adapter-I350-T4-PCI-Express-I350T4BLK-/302320524446?.


About £600 in total without any additional fans.
 
Upvote 0 Downvote
Brilliant!

The final list + i350 ~ £555.00.

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i3-7350K 4.2GHz Dual-Core Processor (£147.80 @ Alza)
CPU Cooler: CRYORIG - M9i 48.4 CFM CPU Cooler (£25.98 @ CCL Computers)
Motherboard: ASRock - Z270M Pro4 Micro ATX LGA1151 Motherboard (£105.91 @ BT Shop)
Memory: Corsair - Vengeance LPX 8GB (2 x 4GB) DDR4-3000 Memory (£63.60 @ Aria PC)
Storage: Intel - 320 Series 40GB 2.5" Solid State Drive (£35.99 @ Novatech)
Case: Thermaltake - Core V21 MicroATX Mini Tower Case (£53.99 @ Amazon UK)
Power Supply: Silverstone - Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (£81.60 @ Kustom PCs)
Total: £514.87
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-05-30 13:44 BST+0100

Please do review and let me if all looks good. I will close this then.

Thanks sir!
 
Upvote 0 Downvote
That's a good catch. Thank you.

I guess all clear now!

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i3-7350K 4.2GHz Dual-Core Processor (£147.80 @ Alza)
CPU Cooler: CRYORIG - M9i 48.4 CFM CPU Cooler (£25.98 @ CCL Computers)
Motherboard: ASRock - Z270M Pro4 Micro ATX LGA1151 Motherboard (£105.91 @ BT Shop)
Memory: Corsair - Vengeance LPX 8GB (2 x 4GB) DDR4-3000 Memory (£63.60 @ Aria PC)
Storage: Sandisk - SSD PLUS 120GB 2.5" Solid State Drive (£48.27 @ Amazon UK)
Case: Thermaltake - Core V21 MicroATX Mini Tower Case (£53.99 @ Amazon UK)
Power Supply: Silverstone - Strider Gold 450W 80+ Gold Certified Fully-Modular SFX Power Supply (£81.60 @ Kustom PCs)
Total: £527.15
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-05-30 14:01 BST+0100

Thanks.
 
Upvote 0 Downvote


SSri,

In my view, for this use, the i7-7700K emphasizes the wrong processor quality, that of the single thread performance, whereas the core /thread count will more advantageous. I'm not experienced with VM's, but colleagues that run them suggest being able to assign a core for every VM and, if possible, 10GB of RAM each. A VM system has some resemblance to a server having a series of workstation nodes, but built into a single chassis.

For this uses, my suggestion that a single workstation/server configuration having 8-cores or more up to a minimum 3.3GHz- Turbo speed, without overclocking, error correcting RAM, and running higher single and double precision will be of a benefit to the performance and stability of several simultaneous VM.s.

You didn't mention a budget, but if the listed system is a guide, my recommendation is to consider a used workstation with an 8-core processor. An example is the system we use for simulation and analysis.

Purchased for US $171:

HP z620_1[/b] (Original) Xeon E5-1620 (4-core @ 3.6 /3.8GHz) / 8GB (1X 8GB DDR3-1333) / AMD Firepro V5900 (2GB) / Seagate Barracuda 750GB + Samsung 500GB + WD 500GB
[ Passmark System Rating= 2408 / CPU= 8361 / 2D= 846 / 3D = 1613 / Mem =1584 / Disk = 574 ] 7.13.16

After upgrades totaling about $1,100

HP z620_1 (2012) (Rev 3) 2X Xeon E5-2690 (8-core @ 2.9 / 3.8GHz) / 64GB DDR3-1600 ECC reg) / Quadro K2200 (4GB) + Tesla M2090 (6GB) / HP Z Turbo Drive (M.2 256GB) + Samsung 850 Evo 250GB + Seagate Constellation ES.3 (1TB) / 800W / Windows 7 Professional 64-bit > > HP 2711x (27" 1980 X 1080)
[ Passmark System Rating= 5675 / CPU= 22625 / 2D= 815 / 3D = 3580 / Mem = 2522 / Disk = 12640 ] 9.25.16 Single Thread Mark = 1903
[ Cinebench R15: CPU = 2209 cb / Single core 130 cb / OpenGL= 119.23 fps / MP Ratio 16.84x] 10.31.16

The total cost was about $1,400 or £1090, probably just over 10% of the cost new. Each E5-2690 cost $2,050 new, 64GBod RAM was a $1,900 upgrade, and a new Tesla M2090 was $2,400.

I would however, recommend using the E5-1600/2600 v2 series motherboard (Bootblock date 2013) as the v2's have higher clock speeds and improved single-thread performance.

For your use, I would recommend the new Quadro P1000 4GB which will support up to 4X 4K DP displays Also, arrange the system to start with 64GB of RAM, but consider using 4X 16GB modules to allow for a possible 128GB.

Notice the increase in compute rate corresponding to core count. The Tesla M2090 6GB coprocessor is not included in the Passmark CPU Mark. . If the budget is strict, the E5-2680 2.7/3.5GHz is also a consideration:

HP Z620 Workstation E5-2680 8-Core 2.7/3.5GHz / 64GB (8X 8GB) > £385.45 + £135.30 post from US

The convenience of this kind of system is that it may be configured using a single CPU and a second processor may be added later by the addition of a 2nd CPU riser, if necessary. Also, the OS in effect includes the OEM version of Windows which will activate automatically.

As each E5 processor contributes 40 PCIe lanes, a single CPU system can easily support an x16 GPU+ x4 PCIe/M.2 SSD + RAID controller + quad network card. Ryzen 7 supports only 20 PCIe lanes and would be sharing lanes for most of the peripherals.

I should mention that we have had quite a number of upgraded, used workstations since 2009 and have never had a single component failure. Some of these systems have run simulations at full load for several days, and full days of CPU rendering at 100% CPU utilization. The HP z420 and z620 systems have also been particularly quiet.

The latest system uses a new HP z620, case /chassis/power supply, CPU, and GPU combined with a used motherboard and drives transferred:

HP z620_2 (2017) > Xeon E5-1680 v2 (8-core@ 4.1GHz) / 64GB DDR3-1866 ECC Reg / Quadro P2000 5GB / HP Z Turbo Drive M.2 256GB + Intel 730 480GB + Seagate Constellation ES.3 1TB / ASUS Essence STX PCIe sound card / 825W PSU / Windows 7 Prof.’l 64-bit > 2X Dell Ultrasharp U2715H (2560 X 1440) / Logitech z2300 2.1 Sound

[Passmark Rating = 6166 / CPU rating = 16934 / 2D = 820 / 3D= 8849 / Mem = 2991 / Disk = 13794] 4.24.17 Single Thread Mark = 2252

Total cost was about $1,900 (£1480)

If you knew that an 8-core would remain sufficient, there are performance advantages to a single processor as there is no parity check sync between dual processors (registered RAM).

The new 3D modeling system does that. The following was assembled using a new HP z620 case/chassis, power supply, CPU and GPU combined with a used z620 E5-v2 motherboard, and drives transferred from the previous system. As this includes 3D modeling in it's uses, the single thread rating was more important.

If the budget is more generous- in the £4000+ range, the final option is to consider building a system based on a https://www.supermicro.com/products/system/tower/7038/SYS-7038A-i.cfm which provides a case/chassis/CPU coolers/power supply and the user only plugs in the CPU's, RAM, GPU, and drives. These are built by servers specialists and have high performance, ultra-reliable components. the motherboard in the SYS 7038-! supports 2TB of RAM and three double height GPU's. that one uses LGA2011-3 Xeon 2600 v3 and v4 CPU's.

Interesting project!

Cheers,

BambiBoom




 
Upvote 0 Downvote
@BambiBoom.

Thank you very much for a detailed recommendations for running multiple VMs. The initial i7 spec was never intended to run all VMs but potentially run PfSense-VPN only

As recommended by USAFRet, I have decided to split this into two boxes, one physical machine running PfSense on an i3 and the second to be specified soon to run other VMs (2) to (5) in my first post. This, I believe, would be better as I definitely prefer keeping the security box independent of VMs.

I have not thought of any budget. PfSense box is going to consume about £600. I am not sure, if I can get the best in another £1400-£1500. I do not need displays. I have 3 at home, while I can always hook up the server on my Full HD TV!.

BTW, I am not sure I should import it from the US, for reasons of import duties (may be 20%) and postal charges of any returns.

I do want get a couple of used E5 V2 Xeons and go from there. Although new (v4) or (to be v5) is better technology and architecture wise, I am happy to scoop a low value Xeons from ebay. I intend taking this as an independent project and seek advise on this forum. Your recommendations are valuable to me. I thank you very much for this. Unfortunately, I cannot pick both yours and USAFRet as solutions. I am going to have to pick one of them. I am sorry, if I do not choose one of you!



 
Upvote 0 Downvote


SSri,

I was seeing both applications as a find of server / node configuration as the functions might effectively share a larger pile of RAM. But, separating the two functions is the proper arrangement. USAFRet always makes very sensible recommendations.

I did want to make the suggestion of a used multi-core and potentially dual Xeon E5 workstation for the VM system and to recommend Xeon E5 v2's as having the best core count to PCIe lanes to memory bandwidth to clock speed to performance to cost ratios. The v2 are only now becoming reasonably priced.

If you every ever need a fast, inexpensive banger system: HP z420 / E5-1650 v2 6C@ 3.5/3.9GHz and Quadro K620.

While prices in the US are less, the post and import fees can make them as expensive as in the UK. But, as the UK is still also the EU, I see very good choices on UK, German, and Italian Ebay. They do cost more, but in general EU examples appear to be closer to a ready to use condition. In the US, used workstations always have all the best components taken out and end up with a dog's dinner of ancient GPU's and drives. When I bought the HP z620_1 for $270 it had three HD's and one was a Seagate Barracuda from 2006, the GPU was a GT 9800- from about the same geological stratum.

It's an interesting project and I'd enjoying hearing of your progress.

Cheers,

BambiBoom

 
Upvote 0 Downvote
You must log in or register to reply here.

TRENDING THREADS

Latest posts

Moderators online

Share this page

COMPANY
RESOURCES
FOLLOW
Top Bottom