Question Monster server for 3TB RAM application/workstation or terrible idea?

[J-dubya]

I currently use a machine that I built a year ago. It has the 32 core AMD Threadripper, 256GB of RAM, 60 something TB of HDD (mostly M.2), and Windows 10 Pro OS. This is too slow for the timeline I'm trying to work with and not nearly enough RAM to have the accuracy I would like. It currently takes about 20 days to get through a set of solves for evaluation and I am limited on the size of the game tree due to RAM.

I am considering buying a Dell Poweredge R930 with 4 CPUs: E7-8867v4 2.4GHz 72-Cores total, 3TB RAM. I'm not interested in buying the Windows Server license for all 72 cores so I would be using Linux Ubuntu. The solver application will work well on that OS I hear. I've never used Linux OS, but am somewhat savvy with tech stuff (built multiple PCs, did some coding in college).

Many people that run this application rent servers like this for $1200 a month and just use it 1 month or 2. I will need to use it for several months and much more in the future. It seems like it makes more sense to just buy the server. I know it will be loud. I know it will use TONS of power.

Has anyone messed with something like this? What kind of issues will I be running into? How hard is it for a Linux newbie to set up Ubuntu with a decent GUI (dont need much in the way of graphics, just a GUI to interact with the solver app)? Is it possible to change out the fans in a big server like this to quieter fans? The fan plugs are not standard 4 pin, could possibly modify plugs..? Is there a better OS for this server to do what I'm trying to do? Is it a completely insane idea to start with? Are there builds that would work better, considering I want 3TB of RAM and at least 60 CPU cores (preferably more)?

Thanks!

 

[kanewolf]

I currently use a machine that I built a year ago. It has the 32 core AMD Threadripper, 256GB of RAM, 60 something TB of HDD (mostly M.2), and Windows 10 Pro OS. This is too slow for the timeline I'm trying to work with and not nearly enough RAM to have the accuracy I would like. It currently takes about 20 days to get through a set of solves for evaluation and I am limited on the size of the game tree due to RAM.

I am considering buying a Dell Poweredge R930 with 4 CPUs: E7-8867v4 2.4GHz 72-Cores total, 3TB RAM. I'm not interested in buying the Windows Server license for all 72 cores so I would be using Linux Ubuntu. The solver application will work well on that OS I hear. I've never used Linux OS, but am somewhat savvy with tech stuff (built multiple PCs, did some coding in college).

Many people that run this application rent servers like this for $1200 a month and just use it 1 month or 2. I will need to use it for several months and much more in the future. It seems like it makes more sense to just buy the server. I know it will be loud. I know it will use TONS of power.

Has anyone messed with something like this? What kind of issues will I be running into? How hard is it for a Linux newbie to set up Ubuntu with a decent GUI (dont need much in the way of graphics, just a GUI to interact with the solver app)? Is it possible to change out the fans in a big server like this to quieter fans? The fan plugs are not standard 4 pin, could possibly modify plugs..? Is there a better OS for this server to do what I'm trying to do? Is it a completely insane idea to start with? Are there builds that would work better, considering I want 3TB of RAM and at least 60 CPU cores (preferably more)?

Thanks!

Is this for a commercial (money making) situation? If so, then there can be tax advantages to renting/leasing the hardware compared to a capital purchase. You should consult a tax professional if this is the case.

 

[J-dubya]

Thanks! It will not be for commercial use though.

 

[kanewolf]

I currently use a machine that I built a year ago. It has the 32 core AMD Threadripper, 256GB of RAM, 60 something TB of HDD (mostly M.2), and Windows 10 Pro OS. This is too slow for the timeline I'm trying to work with and not nearly enough RAM to have the accuracy I would like. It currently takes about 20 days to get through a set of solves for evaluation and I am limited on the size of the game tree due to RAM.

I am considering buying a Dell Poweredge R930 with 4 CPUs: E7-8867v4 2.4GHz 72-Cores total, 3TB RAM. I'm not interested in buying the Windows Server license for all 72 cores so I would be using Linux Ubuntu. The solver application will work well on that OS I hear. I've never used Linux OS, but am somewhat savvy with tech stuff (built multiple PCs, did some coding in college).

Many people that run this application rent servers like this for $1200 a month and just use it 1 month or 2. I will need to use it for several months and much more in the future. It seems like it makes more sense to just buy the server. I know it will be loud. I know it will use TONS of power.

Has anyone messed with something like this? What kind of issues will I be running into? How hard is it for a Linux newbie to set up Ubuntu with a decent GUI (dont need much in the way of graphics, just a GUI to interact with the solver app)? Is it possible to change out the fans in a big server like this to quieter fans? The fan plugs are not standard 4 pin, could possibly modify plugs..? Is there a better OS for this server to do what I'm trying to do? Is it a completely insane idea to start with? Are there builds that would work better, considering I want 3TB of RAM and at least 60 CPU cores (preferably more)?

Thanks!

Your other questions
Fan swap? Probably not. Servers are designed for very specific airflow and for the capability to survive failures. They don't tolerate non-OEM fans.
The 930 is fairly old, released in April 2015. Memory is the first likely failure item. Buy spares. Next is power supplies and fans. See if you can get it will all four power supplies populated.

 

[USAFRet]

Since this looks to be a very niche use case, what do others do with this application?

 

[J-dubya]

Your other questions
Fan swap? Probably not. Servers are designed for very specific airflow and for the capability to survive failures. They don't tolerate non-OEM fans.
The 930 is fairly old, released in April 2015. Memory is the first likely failure item. Buy spares. Next is power supplies and fans. See if you can get it will all four power supplies populated.

Thank You! I was afraid that would be the case with the fans.

 

[J-dubya]

Since this looks to be a very niche use case, what do others do with this application?

Most other people that use this application to the extent that I am rent servers similar to what I am buying. Someone else sets up the server and they just install the app and use it. That is definitely the easier way. The monthly fee adds up though and I will need to run the app for many months now and down the road.

Also, most people don't use the app to the extent that I am and so a typical workstation will do just fine for them.

 

[ex_bubblehead]

Where do you intend to run such a behemoth? This isn't something that you can just plug in in the back bedroom.
Have you actually looked at the cost of 3TB of RAM and all 4 CPU's for this machine? You don't just pick these up at the local computer market.

In addition you have to run a server class OS. Ubuntu Desktop isn't going to cut it here. If you really want Ubuntu then you'll need Ubuntu Server (No GUI). Though, for the price you'd be paying for this behemoth, the cost of a Windows Server license would be dwarfed.

 

[USAFRet]

Most other people that use this application to the extent that I am rent servers similar to what I am buying. Someone else sets up the server and they just install the app and use it. That is definitely the easier way. The monthly fee adds up though and I will need to run the app for many months now and down the road.

Also, most people don't use the app to the extent that I am and so a typical workstation will do just fine for them.

On a smaller scale, what experience do you have with this unknown application?

If most other people rent server space for this, maybe that IS the most cost efficient way to do it.

 

[J-dubya]

Where do you intend to run such a behemoth? This isn't something that you can just plug in in the back bedroom.
Have you actually looked at the cost of 3TB of RAM and all 4 CPU's for this machine? You don't just pick these up at the local computer market.

In addition you have to run a server class OS. Ubuntu Desktop isn't going to cut it here. If you really want Ubuntu then you'll need Ubuntu Server (No GUI). Though, for the price you'd be paying for this behemoth, the cost of a Windows Server license would be dwarfed.

Thanks! I do have the needed power supply/breaker/outlet for this server available at my home. I was once an industrial electrician by trade.

The server listed above, with the CPUs and 3TB of RAM installed, is available to me for around $6k. Yes, I realize it will need to be Ubuntu Server. I know I will have to add a GUI to it.

I am trying to find out what issues I will run into with adding a GUI to that OS and how hard it will be to set up. I have not messed with much Linux OS, esp not the server OS.

 

[USAFRet]

Adding a GUI to a Linux Server instance?
Thats a big project in itself.

 

[kanewolf]

Thanks! I do have the needed power supply/breaker/outlet for this server available at my home. I was once an industrial electrician by trade.

The server listed above, with the CPUs and 3TB of RAM installed, is available to me for around $6k. Yes, I realize it will need to be Ubuntu Server. I know I will have to add a GUI to it.

I am trying to find out what issues I will run into with adding a GUI to that OS and how hard it will be to set up. I have not messed with much Linux OS, esp not the server OS.

Does your software work in a cluster or only on a single system?

 

[J-dubya]

On a smaller scale, what experience do you have with this unknown application?

If most other people rent server space for this, maybe that IS the most cost efficient way to do it.

I have been using the application for years on big workstations and recently on my new Threadripper set up.

The difference with me and others is that I will need to run the app for many months. Most that use it, run it for a coupkle months and then they are done. I am developing a more complex strategy that will require many months of use now and more in the future. So it seems more cost effective to own the server.

Just trying to work out the details of getting it up and running, and determine if it is feasible.

 

[USAFRet]

OK...we're just trying to work through the details. asking questions.

Right now, there is exactly one person in this conversation that knows what you're working with.

 

[kerberos_20]

ubuntu has x11 server which is no gui, but you can forward it to remote X client gui, that would save you server resources instead of installing desktop on it

Ubuntu: X11 forwarding to view GUI applications running on server hosts | Fabian Lee : Software Engineer

Although server hosts typically have no graphical desktop and only serve console-based clients, these machines still have the ability to serve a GUI display screen to a remote desktop if necessary. The X11 protocol makes it possible to send the graphical display to a remote graphical desktop...

fabianlee.org

 

[J-dubya]

Adding a GUI to a Linux Server instance?
Thats a big project in itself.

Uhg.. Is it possible to get any other, better OS on a server like that, without spending another $2k to license all the cores?

 

[J-dubya]

OK...we're just trying to work through the details. asking questions.

Right now, there is exactly one person in this conversation that knows what you're working with.

Sorry. I greatly appreciate your help! I mean no offense.

 

[J-dubya]

Does your software work in a cluster or only on a single system?

I am not 100% sure. I believe a single system only though.

 

[USAFRet]

All your questions above are why people contract this out to a server farm, rather than trying to build a system and self host it.

 

[J-dubya]

Overall, this is a very unique situation where old, outdated servers have the perfect hardware setups. Current high performance workstations do ok. That is what I have now, but am limited by the amount of RAM you can put in them, These old servers are also affordable compared to buying or renting the new versions of servers. The big issue is getting them to work the way I need them to (like a workstation). I am positive I need the hardware listed to run the app the way I want to run it. Just trying to figure out how to make it all work and if that is even possible with my limited experience.

 

[J-dubya]

ubuntu has x11 server which is no gui, but you can forward it to remote X client gui, that would save you server resources instead of installing desktop on it

Ubuntu: X11 forwarding to view GUI applications running on server hosts | Fabian Lee : Software Engineer

Although server hosts typically have no graphical desktop and only serve console-based clients, these machines still have the ability to serve a GUI display screen to a remote desktop if necessary. The X11 protocol makes it possible to send the graphical display to a remote graphical desktop...

fabianlee.org

I see. Thank you for the link. That is a bit more complex than I was hoping to go. Is there another OS that would work better for getting this server to act somewhat like a workstation?

 

[USAFRet]

Is there another OS that would work better for getting this server to act somewhat like a workstation?

Linux server or Windows server.

Is it OK to spend $2k to learn Linux?

 

[J-dubya]

Linux server or Windows server.

Is it OK to spend $2k to learn Linux?

More than likely, yes. I doubt this will be the last project like this for me. Linux seems to be a much better option for OS cost-wise for servers. Which version of Linux Server would you recommend for this?

 

[kanewolf]

I am not 100% sure. I believe a single system only though.

The reason I ask is because when you have a hardware problem, it doesn't stop everything. The rest of the cluster continues to process and will pick up the failed jobs. Much more resillient than a single server. If it was me, I would investigate if that is an option.

 

[J-dubya]

All your questions above are why people contract this out to a server farm, rather than trying to build a system and self host it.

I understand. I don't mind a learning curve. I'm just trying to understand what to expect and if it is worth that learning curve.