Recent content by JRRT

My point was more that these models are run on GPUs in the first place because the way that they operate makes them constrained primarily by the speed of memory access. Video memory is both fundamentally faster in a serial bit transfer sense, and built with more lanes of access. I commented...

I am aware of pretty much everything here, and unless you are a statistition I probably have more experience in the field. I did mention being a biologist. So yes, I am aware of confirmation bias. But at this point I am fairly sure that we are talking past each other. Still, I will...

Thank you for your thoughts.

Did you ever do this? Did it work? I can't really afford a server motherboard and processor at the moment, but I need a workaround to be able to run some very memory demanding simulations. Even if it takes a couple months to finish, it would at least be possible with this much memory if it...

So I could easily be wrong. But have you looked at this analysis: https://www.efinancialcareers.com/news/2023/06/which-programming-language-uses-the-most-energy

I am showing my own ignorance, but I have a few questions. Are you the one posting questions here because you can read and write English and the IT guy doesn't, are you his assistant, or something else? Because it would be nice to get specific information directly from the person responsible...

Kudos for helping the guy understand what he needs to do, rather than just telling him he needs to hire an expert.

Very different from my experience, but I agree that it makes more sense to do it this way for a situation like you described. I probably need to get up to date on current rules and best practices. Thank you.

Actually, there is a llama implementation that can use SSDs for a swap file. I think that it's either Kobold or Oogabooga. I suppose that this could be a less bad option, but really at this price for that purpose, even for a cheapskate like me it seems to make more sense to get 128+ GB of DDR...

Perhaps. I don't know if Amazon or Google are the best options when I don't want to use their hosting, but maybe I am not aware of the possibilities, and HP also has some possibilities.. Dell certainly has some very nice server stuff. I did speak with an IBM rep, but at the time they were...

Thank you very much for the link suggestions. I am going to see what I can find.

Wait, your retired Air Force, right? So you probably already know that you can not have a manufacturer come in, and certainly can't send whole systems out, when classified data is involved. I know that you all handle repairs in house for classified systems, because of the fact that unlike...

Very true. But most companies are not going to want to support these situations anyway. One client is developing not just a new PCIe card, but ASICs to go on it. If you know a server maker where that does not void the warranty, I would love to know. Eventually, of course, it's going off to...

I don't know how to say this without sounding like a jerk, but would you pay the extra money, or would a company that you are working for and who's management doesn't even know that there are cheaper options? I am not a big company, and I don't have any clients with money to spare (yet, it...

So I am personally interested. I am working on several different projects. For myself and clients. I can't go into excessive detail, but I think that I can explain enough that it should make sense. One of the reasons that a number of the AI models out there are so resource intensive is that...