Question Is it possible to have AMD GFX card to use AI tools?

[apalace]

It seems I am getting really old, and the technology is running faster than me..

I have RTX 3070 Ti with only 8 Gigs of vRam. I installed some AI tools to use locally, but this tools need CUDA to be installed and are hungry for VRAMS, therefore I want to upgrade to a card with at least 24 gigs of vrams..
I have one option here from nVidia: RTX 4090 which costs over €1700 which is really expensive.

I looked into AMD cards, like RX 7900xtx, and it has 24 gigs and I can save €700. But then I have to get rid of AI tools that I like to use locally on my machine.

The question is:
Is there now a solution for that? I mean, can we use CUDA with AI Tools while having AMD cards instead of nVidias? Or am I still stuck with nVidia cards to achieve that?
Any ideas, comments, solutions, possibilities... etc?

P.S.: I mean for Windows 11

Very thankful for any of the above!

 

[kanewolf]

It seems I am getting really old, and the technology is running faster than me..

I have RTX 3070 Ti with only 8 Gigs of vRam. I installed some AI tools to use locally, but this tools need CUDA to be installed and are hungry for VRAMS, therefore I want to upgrade to a card with at least 24 gigs of vrams..
I have one option here from nVidia: RTX 4090 which costs over €1700 which is really expensive.

I looked into AMD cards, like RX 7900xtx, and it has 24 gigs and I can save €700. But then I have to get rid of AI tools that I like to use locally on my machine.

The question is:
Is there now a solution for that? I mean, can we use CUDA with AI Tools while having AMD cards instead of nVidias? Or am I still stuck with nVidia cards to achieve that?
Any ideas, comments, solutions, possibilities... etc?

Very thankful for any of the above!

CUDA is an NVIDIA tailored language. There is apparently a workaround -- https://wccftech.com/nvidia-cuda-directly-run-on-amd-gpus-using-scale-toolkit/
I have no idea how well the SCALE interface layer works.

 

[apalace]

CUDA is an NVIDIA tailored language. There is apparently a workaround -- https://wccftech.com/nvidia-cuda-directly-run-on-amd-gpus-using-scale-toolkit/
I have no idea how well the SCALE interface layer works.

Awesome, thank you!
I just hope, that someone with a real experience who uses AMD cards with this SCALE to confirm it is really working as a CUDA, or any solution that can open the door for me to upgrade from nVidia card to AMD one.

 

[COLGeek]

Depends on the tools you are using. For example, when I first started experimenting with Ollama, it required a Nvidia device for GPU use. Now, it also supports select AMD GPUs.

https://ollama.com/blog/amd-preview

 

[apalace]

Depends on the tools you are using. For example, when I first started experimenting with Ollama, it required a Nvidia device for GPU use. Now, it also supports select AMD GPUs.

https://ollama.com/blog/amd-preview

All sort of different tools that come out for different usages, as example:
ComfyUI , SwarmUI , XTTS RVC UI , Rope Live Stream , LivePortrait , Kohya_GUI , Deep-Live-Cam , MMVCServerSIO , hallo ... etc.

 

[COLGeek]

All sort of different tools that come out for different usages, as example:
ComfyUI , SwarmUI , XTTS RVC UI , Rope Live Stream , LivePortrait , Kohya_GUI , Deep-Live-Cam , MMVCServerSIO , hallo ... etc.

Understood. You'll need to assess the feasibility of each to determine whether an AMD GPU is suitable for them or not.