Someone squeezed a 28.9M LLM onto an ESP32-S3, and so can you

Credit: slvDev_ Published Jul 23, 2026, 9:46 PM EDT Simon is a Computer Science BSc graduate who has been writing about technology since 2014, and using Windows machines since 3.1. After working for an indie game studio and acting as the family's go-to technician for all computer issues, he found his passion for writing and decided to use his skill set to write about all things tech. Since beginning his writing career, he has written for many different publications such as WorldStart, Listverse, and MakeTechEasier. However, after finding his home at MakeUseOf in February 2019, he would eventually move on to its sister site, XDA, to bring the latest and greatest in Windows, Linux, and DIY electronics. Sign in to your XDA account Summary ESP32-AI fits a 28.9M-parameter LLM on an ESP32-S3 that can tell coherent stories. It uses emory-mapped flash for a 25M-param embedding table, and it only needs a ~560K dense core in fast RAM. It runs on-chip fully at ~9 tokens/sec, impressive given the ESP32's tiny size and RAM limits. Ever since LLMs hit the public eye, people have tried to cram them into the smallest thing they can find. We've seen people use both Raspberry Pis and ESP32s to run an AI locally, with varying results. Now, someone has managed to squeeze a 28.9M LLM on an ESP32-S3, and it can tell a pretty good story. ESP32-AI adds a 28.9M LLM to your microcontroller It's very impressive for the chip size On the ESP32 subreddit, user slvDev_ managed to get an LLM running on an ESP32-S3. It can store 28.9 million parameters; for context, the very first version of ChatGPT had 117 million parameters. As such, this ESP32 project only has about a quarter of the smarts of OpenAI's first model, but given how this thing was squashed onto an ESP32 device, that's a far more impressive feat than it might sound at first. ​​​​​ So, how did they manage to get all of those parameters on an ESP32-S3? Well, here's how they explain it: The trick (Gemma's Per-Layer Embeddings): most of a language model's params are a big embedding table you read from, not compute on. So I keep that 25M-row table in flash (memory-mapped XIP) and read only ~6 rows per token (~450 bytes). Only a ~560K dense core needs fast memory. The 25M-param table is basically free to run, it just sits in flash and gets sampled a few rows at a time. The end result is an AI that can tell a story and keep track of the variables, which is pretty great for such a tiny device. If you'd like to download it and run it on your own hardware, head over to the ESP32-AI GitHub page for all the details.