Question AI powered Nvidia Quadro graphics cards ?

[oliveria]

hello good people, I'm about to get into the market to replace my ageing mobile workstation, I've been trying to compare a Quadro A3000 against an RTX 3000 and a P4200, I read about the tensor cores and how they're dedicated to AI , which the P4200 lacks.

Ok, I'm new to the AI side of things and I want to know how buying a mobile workstation with an AI powered Quadro will impact my life or how I can take advantage of the tensor cores , in its basic form for example when two people are using ChatGTP or Gemini , how will the one having a laptop with Quadro A3000 have an advantage over even somebody using a laptop with no graphics card at all in the search results that chatgpt gives.

Or can somebody can explain to me how I will benefit on the AI side of things if I go with for example A3000 and leave a machine with P4200, and A3000 is also having fewer tensor cores compared to RTX 3000 despite being newer and running on 8nm lithography. Are there any hidden secrets in those tensor cores number comparison? I will gladly appreciate your replies.

 

[Eximo]

AI that you are running locally would perform faster, or allow for larger datasets to be loaded. Most AI that is easily accessible is externally run on massive servers, so your system does not matter in those cases.

Some newer CPUs come with NPUs which are portions of the CPU reserved for the type of math that AI tasks benefit from. This is to gain efficiency, not so much performance. And there is little using it today, but maybe soon. Things like Microsoft Recall was one idea, but that just seems like a bad idea overall.

Each generation of Tensor and Ray Tracing cores are different and can't be directly compared. Each generation is faster as far as I know. So a smaller, newer, GPU may just get the job done more efficiently. That is where a very specific task would need to considered and benchmarks looked at for comparison.

The local GPU can be used for loading or training Large Language Models that make up consumer AI, these include things like image generation, video generation (potentially), having a local chatbot, and a few other AI related tasks.

More common uses for GPUs would be standard video encode/decode (either converting, video, compressing video, or decoding live stream videos in your browser), Photo Editing (filters, and effects), 3D graphics (Games), 3D Modeling (AutoCAD/Solidworks/Rhino/Unreal/Unity/etc), Video conferencing (Background replacement), Audio Conferencing (Assisted sound filtering, noise cancellation), Mathematics modeling and calculation, Medical research.

 

[oliveria]

Great, thanks a lot for the nice explanation, so as long as I'm having a quadro that is AI enabled with tensor cores I'll be having nice time doing things with the likes of chatgpt or copilot and others like gemini, I'll consider going for something like Quadro RTX A3000 instead of RTX 3000, hope this will help

 

[Eximo]

Great, thanks a lot for the nice explanation, so as long as I'm having a quadro that is AI enabled with tensor cores I'll be having nice time doing things with the likes of chatgpt or copilot and others like gemini, I'll consider going for something like Quadro RTX A3000 instead of RTX 3000, hope this will help

Not sure you understood.

So a lot of those tools are online, the processing is done on remote servers. They don't run on your local machine at all.

You can get the open source versions of chatGPT, LLama, etc and run them yourself, with some setup. But that would be a separate tool where you maintain the data input and training. You are then limited by the VRAM amount on the GPU as to what datasets can be loaded.

 

[oliveria]

ok and lets say since most people do access AI online via servers, even those who have pc with no graphics card, is there really any difference between me and sb who doesn't have a graphics card when two of us are using AI via online servers, or how can I take advantage if a quadro cad like rtx A3000 and be ahead of sb who has a PC with no graphics card, on the AI side of things

 

[Eximo]

I'm not really sure how to add anything additional here.

If you are using online services, whatever hardware and software they are running is going to be equivalent to all users. There may be price tiers and other such things, different services of course.

Once again, the GPU is capable of running AI tools. You just have to set them up. Your experience will depend on the model you use, the data you provide to it, and how long it takes will depend on the speed of the GPU. Some models will be limited by the amount of VRAM you have. So an 8GB model is going to be less 'knowledgeable' or accurate than a 16GB or 24GB model. Highly depends on the model you load and what you are trying to accomplish.

I can't speak to the future much, but at some point there may be local AI options made available in Windows to general hardware, but we aren't there now.

Windows specifically has very strict requirements for their local AI stuff. That means a brand new CPU Either the Intel 200 series, Qualcomm Snapdragon, or AMDs latest two generations with AI in the CPU name. They should start seeing some local AI tools that don't require user setup in the next little while. However, by the time there is something standardized we will be well beyond these first generation products and Nvidia will probably be at least two or three GPU architectures ahead of Ada. (Blackwell should launch early next year)