llm

Hello everyone! I have an old server from 2013 that has a tesla M40 gpu, im trying to run llms locally on that gpu through ollama but haven't had luck, I know its not officially supported since it has a compute capability of 3.5 but I tried using a modified version of ollama but still it falls...

I'm interested in building a system initially with 2x 4090 GPUs but with the ability to add two more later if the need arises. This is for running LLMs and perhaps fine tuning small ones. So far I have found a very limited number of motherboards that have more than 3 PCI slots, and of those...

My use case is I have installed LLMs in my GPU using the method described in this Reddit Post. I want to run WizardLM-30B, which requires 27GB of RAM. (I also have a quantized version with lower RAM requirements, but I do not have a number on the RAM requirements. I just know that I can fit...

Hello, I would appreciate some guidance on what hardware (GPU or otherwise) I should purchase to enable me to run LLMs locally on my machine. Here are my system specs. CPU: AMD Ryzen 9 7950X Motherboard: ASRock X670E PG Lightning AM5 ATX Mainboard. RAM...