- Dec 11, 2024
- 120
- 46
- 120
I tried using AMD ROCm software on my RX 6750 GRE 12G GPU in early 2024 for running ChatGLM3-6B, and it was a total a disaster:
The installing steps are annoying even for me, a CS graduate with some experience for every espect of CS.
I followed instructions provided by AMD, successfully installed ROCm 5.8(the latest version with torch support at that time) and torch(rocm ver), and then I found that nothing works fine, I need to setup a lot more (environment variables not mentioned by AMD's doc and not automatically setted by the install process). ROCm is complex to setup, especially on a rare GPU, I finally mixed the recommended settings of 6750xt from the document and the actual GPUID I found by myself. Then another problem comes out: I use 7700x which contains a iGPU, and ROCm needs to disable it at that time(maybe this issue is fixed later), if not doing so, everything'll break down without noticable error reports.
I noticed that Intel may cancel the Arc Pro series and release B580 24G later in 2025, so I'm wondering if Intel provides some software stacks like ROCm that I can use it OUT OF THE BOX? The performance is also not fine when using 6750GRE12G(I know 6000-series AMDGPU lacks AI performance), but it's acceptable, and 12GB vram makes it enough to run ChatGLM3-6B-int8——I don't care about performance&convenience a lot, so a 24GB version of B580 is attractive to me.
I already have three Arc GPUs: A380, A770-8G, B580 (just give it away to my friend). If someone can tell me that Intel's AI software stack is easier than ROCm to deploy, I may try it later 😉
The installing steps are annoying even for me, a CS graduate with some experience for every espect of CS.
I followed instructions provided by AMD, successfully installed ROCm 5.8(the latest version with torch support at that time) and torch(rocm ver), and then I found that nothing works fine, I need to setup a lot more (environment variables not mentioned by AMD's doc and not automatically setted by the install process). ROCm is complex to setup, especially on a rare GPU, I finally mixed the recommended settings of 6750xt from the document and the actual GPUID I found by myself. Then another problem comes out: I use 7700x which contains a iGPU, and ROCm needs to disable it at that time(maybe this issue is fixed later), if not doing so, everything'll break down without noticable error reports.
I noticed that Intel may cancel the Arc Pro series and release B580 24G later in 2025, so I'm wondering if Intel provides some software stacks like ROCm that I can use it OUT OF THE BOX? The performance is also not fine when using 6750GRE12G(I know 6000-series AMDGPU lacks AI performance), but it's acceptable, and 12GB vram makes it enough to run ChatGLM3-6B-int8——I don't care about performance&convenience a lot, so a 24GB version of B580 is attractive to me.
I already have three Arc GPUs: A380, A770-8G, B580 (just give it away to my friend). If someone can tell me that Intel's AI software stack is easier than ROCm to deploy, I may try it later 😉
Facebook
Twitter
Instagram