News Modder crams LLM onto Pi Zero-powered USB stick, but it isn't fast enough to be practical

[Admin]

A Raspberry Pi Zero can run a local LLM using llama.cpp. But, while functional, slow token speeds make it impractical for real-world use.

Modder crams LLM onto Pi Zero-powered USB stick, but it isn't fast enough to be practical : Read more

 

[JRStern]

Some day all USB sticks will host LLMs and they can chat with each other when not in use.

 

[derekullo]

Some day all USB sticks will host LLMs and they can chat with each other when not in use.

I believe that is called a botnet !