Gemma 4 E2B proves Raspberry Pi-sized LLMs don't have to be terrible anymore

Published Aug 2, 2026, 2:00 PM EDT Ayush Pande is a PC hardware and gaming writer. When he's not working on a new article, you can find him with his head stuck inside a PC or tinkering with a server operating system. Besides computing, his interests include spending hours in long RPGs, yelling at his friends in co-op games, and practicing guitar. When it comes to running large language models capable of aiding your productivity, there are plenty of options to choose from. Unfortunately, LLMs with the right reasoning capabilities for real-life tasks can typically only be run on the cloud. If you’re anything like me, you’re probably not too fond of relying on privacy-invasive, subscription-heavy cloud models, leaving local LLM-hosting setups as the cost-effective alternative. Mixture-of-Experts models are a great way to accommodate bulky LLMs with dozens of parameters in VRAM-constrained graphics cards. But you still need enough VRAM + system memory to run MoE models, making them a bit difficult to host on budget hardware. While I wouldn’t go so far as to say that small-sized models can replace their powerful 20B+ counterparts, I recently encountered some LLMs that are good enough for light productivity tasks. To be more specific, I’m talking about the ultralight models in the Gemma 4 lineup, and the E2B variant, in particular, is quite capable even though it’s small enough to fit on a Raspberry Pi. Lightweight LLMs leave a lot to be desired They just aren’t ideal for everyday tasks A couple of weeks ago, I decided to conduct a wild experiment where I ran the agent harness Pi on the Raspberry Pi, and the results were rather mixed until I tried one specific model. Although most 1B-3B models were able to run simple commands, they lacked the reasoning capabilities to understand complex prompts unless I over-explained everything. Qwen 3.5 2B, for example, was good enough for simple workloads, but it failed miserably when I asked it to create an extension for Pi (which is the whole shtick of this agent harness). The same applies to Qwen 2.5 Coder, Qwen 3 4B Thinking 2507, and other low-parameter models. But even leaving this isolated case aside, I haven’t had a lot of luck with sub-7B models when it comes to precision. There’s the time I tried connecting my Home Assistant hub to DeepSeek R1 Distil Qwen 1.7B via an MCP server, and the LLM would toss hallucinated garbage as the output whenever I tried questioning it about my smart devices. I had similar results when I tried to use 2B-4B models to control my Nextcloud hub using MCP tools. However, one encounter with Gemma 4 E2B changed that... It even supports image and audio inputs Compared to most of the LLMs I’ve tried running on my Raspberry Pi, Gemma 4 E2B has delivered significantly better results, and that’s due to its neat design. Technically, Gemma 4 E2B has 5.1 billion active parameters, so you might be inclined to think it’s too heavy for a weak Raspberry Pi system. However, this powerful LLM’s actual computation cost is around 2.3B, making it light enough to run on modern non-Zero Raspberry Pi single-board computers. That’s because a significant fraction of E2B’s parameters are per-layer embedding tables, which remain on storage devices for the most part. Once I start querying the LLM, only the PLE slices needed for the inference operations are mapped into the memory, which brings its effective parameters down to 2.3B from 5B. As for its reasoning capabilities, Gemma 4 E2B also supports image and audio inputs. I’ve tested its vision capabilities by giving it a bunch of photos, and it has identified objects pretty well. It can even identify some elements of my screenshots, though I must admit it tends to falter when the images have obscure apps and tools in the frame. I’ve also tossed PDF documents into this tiny LLM, and it’s surprisingly good at summarizing them. Performance-wise, it holds up decently well on a Raspberry Pi 5. Of course, it’s not as fast as 0.7B models, but its token generation rate of 5.8–6.5 t/s is good enough for light productivity tasks. On the apps front, I’ve connected it to Karakeep and Paperless-GPT, and it’s pretty decent at auto-summarizing videos, articles, and documents. But for anything requiring more precision, I’d stick to Gemma 4 E4B instead. Its E4B counterpart is even better, provided you’ve got a high-end Raspberry Pi Just like Gemma 4 E2B, E4B also uses per-layer embeddings, except it's able to harness a larger 8B knowledge base and has an effective parameter size of 4.5B. Of course, this means you’ll need a Raspberry Pi model with more memory to run the model, on top of dealing with slower inference speeds. However, it’s a lot more capable than its E2B counterpart. On my Raspberry Pi 5 (8GB), Gemma 4 E4B generates roughly 2.5-3 tokens every second, so it’s a tad too slow for normal inference operations. However, it’s the only LLM that managed to create a functioning extension for Pi, and I’ve connected it to Open Notebook as well as Blinko instances as the primary inference model. Heck, I even paired it with a Docker MCP setup, and although the LLM failed at automatically spinning up new containers, it was able to generate the right syntax for it. Plus, it could perform typical management and monitoring operations pretty well, but the Raspberry Pi’s slow inference speeds make that a challenge. As such, I currently stick to Gemma 4 E2B for lightweight inference tasks, while leaving Gemma 4 E4B running on an old laptop for intermediate-level experiments requiring better reasoning capabilities. llama.cpp Llama.cpp is an open-source framework that runs large language models locally on your computer.