News Raspberry Pi 5 successfully accelerates LLMs using an eGPU and Vulkan

[Admin]

A Raspberry Pi 5 hooked up to an AMD Radeon-powered eGPU has been demonstrated using the graphics hardware to accelerate a Large Language Model (LLM).

Raspberry Pi 5 successfully accelerates LLMs using an eGPU and Vulkan : Read more

 

[bit_user]

Adding the RTX 4090 benchmark to the mix (second slide) shows how much LLM performance a powerful modern PC can muster.

I wonder why he doesn't show how fast the same x86-powered PC would run with any of the graphics cards he tested on the Raspberry Pi?

 

[geerlingguy]

I wonder why he doesn't show how fast the same x86-powered PC would run with any of the graphics cards he tested on the Raspberry Pi?

Time. I just haven't had time to install and configure each of the cards on the PC, then get Vulkan support set up and running (ROCm still isn't supported for most of these cards even on x86).

They may perform very slightly better, but probably not much, at least for any models that fit within VRAM.

 

[bit_user]

Time. I just haven't had time to install and configure each of the cards on the PC, then get Vulkan support set up and running (ROCm still isn't supported for most of these cards even on x86).

I understand. Thanks for replying!

They may perform very slightly better, but probably not much, at least for any models that fit within VRAM.

Good to know. If much of the time were spend loading the model onto the GPU or on host-based preprocessing, it would be interesting and enlightening. However, if it's overwhelmingly GPU-dominated, then I'd agree it's probably rather pointless to do the comparison.

As always, thanks for stopping by and keep up the great work!
: )


P.S. did we ever find out why Raspberry Pi didn't officially spec the Pi 5's PCIe as 3.0?