I'm running AI on my NAS at 5 tokens per second, and it's surprisingly useful

Published Aug 2, 2026, 7:30 AM EDT Jeff's been involved in the IT industry since before the Internet and spent more than 20 years working in technical support, system administration, network administration, and consulting roles. He holds an undergraduate degree in English, a Master's degree in English with a focus on professional writing and editing, and another Master's degree in Computing & Information Systems. After teaching university English and computer science for a few years, Jeff launched his writing career. He's written for Macworld, Tom's Hardware, groovyPost, The Mac Observer, and more before beginning here at XDA. I knew this was a bad hardware match before I started. My UGREEN DXP4800 Pro doesn’t have a discrete GPU; it already spends its days as a NAS, and there was no reason to expect CPU-only AI inference to be pleasant. I installed Ollama anyway, connected Open WebUI to it, and pulled down small 4B models to see whether the whole thing was merely possible or actually useful. It turned out to be both, though with a lot more waiting than I would’ve preferred. My NAS had everything except hardware meant for AI Getting Ollama running was the easy part, mostly speaking UGOS doesn’t provide a tidy Ollama app to install, so Docker was the obvious route. I put Ollama in a Compose project, exposed port 11434, then added Open WebUI and mapped port 3000 on the NAS to port 8080 inside the container. That part went smoothly enough that I briefly thought the rest might, too. At least I had a proper browser interface instead of spending the entire experiment in a terminal. I still managed to break the connection between the two containers almost immediately. I pointed Open WebUI at Ollama using HTTPS, even though Ollama wasn’t serving HTTPS there. I then spent a bit staring at a connection refusal before realizing what I'd done. Changing the address to http://ollama:11434 fixed it. It was a small mistake, but exactly the kind of mistake polished cloud services keep you from having to think about. I pulled Qwen3 4B and Gemma 3 4B because trying to cram a much larger model onto this machine would've been missing the point. I wasn't trying to prove the NAS could technically load something huge if I pushed it hard enough. I wanted something small enough that I might leave it installed after the novelty wore off. Even at that size, though, the hardware limits showed up quickly. CPU inference makes every generated word feel expensive Five tokens per second changes how you use AI Qwen3 4B managed about 5.4 tokens per second in my testing. Gemma 3 4B was slower, at around 3.2 tokens per second, and it also loaded CPU-side vision components I wasn't using. Ollama pushed CPU usage to roughly 197% while Qwen was working and used around 3.1GB of memory. On a NAS with about 7.7GB of RAM, I could also see the system leaning hard on swap. Those numbers don't sound completely awful until you're sitting there watching the answer come in. At 5.4 tokens per second, you notice every pause, every unnecessary paragraph, and every moment when the model decides to wander. A response that feels instant in ChatGPT becomes something you can watch being built. I also started making my prompts shorter because every extra detour carried a very real time penalty. The ugliest example came from a Pi-hole troubleshooting test. I gave Qwen backup logs containing a deliberately planted bad dnsmasq option in /etc/dnsmasq.d/99-ai-test.conf and asked it to identify the problem. More than 17 minutes later, it was still chasing NTP issues instead of the bad config file, and I finally stopped the run by restarting Ollama. Waiting that long for a wrong answer is one thing; watching the NAS grind away the whole time made it much harder to shrug off. Small local models can still handle useful everyday work Gemma found the problem while Qwen kept wandering elsewhere I ran the same Pi-hole test through Gemma 3 4B and got a much better result. It found the dnsmasq problem in 4 minutes and 13.44 seconds, which was still slow enough that I had plenty of time to wonder whether it was going off course too. This time, though, it actually found the thing I had broken. It added some advice afterward that I wouldn't trust without checking, but at least the core diagnosis was right. The more important test came when I stopped asking the NAS to act like a reasoning machine. I fed the model rough notes from a Jellyfin Docker setup and asked it to turn them into usable Markdown. It preserved the details that mattered, including port 8096, paths, mounts, and settings, while cleaning up the structure enough that I could actually use the result. That took 1 minute and 49.88 seconds, and I didn't get impatient. That was when the whole setup started making more sense. I don't need a 4B model on my NAS to solve difficult technical problems, do serious research, or replace the stronger cloud models I already use. I do have plenty of rough notes, setup details, half-organized snippets, and text that mostly needs cleaning up. For that kind of work, a minute or two is fine, especially if I can start the job and go do something else. A CPU-only local model works best when you give it bounded jobs with information already in front of it. Formatting notes, cleaning up Markdown, summarizing text, and reorganizing setup details made far more sense on my NAS than asking a 4B model to solve difficult technical problems. Cloud AI is faster and far more capable There is no pretending this competes with ChatGPT today If the only question were which tool gives me the best answer in the least time, this experiment would've ended quickly. Cloud models are dramatically faster, handle complex prompts better, and usually need far less babysitting. I can open ChatGPT, ask a difficult question, and often have something useful before Qwen settles on its first bad theory. The 17-minute Pi-hole detour made that gap hard to ignore. There's also a resource cost that doesn't show up on a subscription bill. Ollama isn't charging me per token, but it's consuming CPU time and memory on a machine that's supposed to be doing other things. During testing, I saw available memory drop to roughly 409MB while swap usage climbed to around 5GB. Nothing crashed, but those numbers were enough to stop me from treating the NAS like an unlimited AI box. Then there's the model choice problem. Qwen was faster at raw token generation but completely missed one troubleshooting test, while Gemma was slower and found the actual fault before adding some questionable recommendations afterward. I can't just pick one model and assume it'll be equally good at everything. That's another bit of friction I don't have to think about when I'm using a strong cloud model. That comparison misses why I keep using it The local model only needs to earn its place Trying to make this NAS compete directly with ChatGPT is where the idea stops making sense. It isn't going to win on speed, model size, reasoning quality, or convenience with difficult prompts. Once I stopped caring about that comparison, the setup became easier to judge. I just needed it to handle boring little jobs on my own network without involving another service. The slowness also made me more selective about what I send to it. I'm not giving this thing a complicated technical mystery if I know I could get a stronger answer from a cloud model in a fraction of the time. I will give it notes that need cleaning up, Markdown that needs organizing, text that needs summarizing, or other work where the information is already there. Those are the jobs I usually delay because they're easy enough to do myself and annoying enough that I don't want to. And because the NAS is already on, using it barely adds any friction. Open WebUI is sitting there in a browser, Ollama is already running, and I don't need to boot another machine or prepare anything first. I can paste something in, switch tabs, and come back when it's done. That mundane convenience is probably the biggest reason the stack survived after I finished testing it. A slow local model can still be worth running My first few hours with Ollama on the DXP4800 Pro confirmed almost every objection I had before I started. The models are slow, CPU usage spikes, memory tightens, and even a small model can spend an astonishing amount of time being confidently wrong. There are much better ways to run local AI, especially if you've got a GPU with enough VRAM. My NAS doesn't, and after testing this setup, I have no interest in pretending otherwise. It’s slow, the setup isn’t elegant, and I still open it every day. What changed was the job I expected the model to do. I don't need my NAS to replace ChatGPT, and after watching Qwen spend 17 minutes looking in the wrong place, I'm definitely not tempted to try. I need it to take a handful of small, repetitive jobs off my plate, using hardware that's already powered on and sitting in my network rack. It's slow, the setup isn't elegant, and I still open it every day. Ollama Released July 3, 2023 Developer(s) Jeffrey Morgan and Michael Chiang Price model Free Even on hardware that may not seem suited for the job, Ollama can be surprisingly useful.