I stopped trying to make one local LLM do everything, and that's when it got useful

Published Nov 10, 2026, 10:31 AM EST Nolen began their writing career in 2019, with three years dedicated to editing the Creative section at MakeUseOf. Their expertise lies at the crossroads of technology and creativity, covering areas like photography, video editing, and graphic design. Outside of work, you'll often find Nolen diving into a good book, writing their own stories, or playing video games. Anyone running local models on a regular gaming GPU ends up making the same trade-off. An 8GB card has room for one decent model at a time, so whichever one is loaded handles every request that comes in that day, whether it suits the job or not. Smaller models tend to work faster but they can really struggle with heavy workloads or multi-step tasks. The obvious fix is loading something a little bigger, but then every quick question gets slower as the session grows. My habit has been swapping my default models up and down a size every week, then blaming whichever one was loaded when its responses were subpar. It took an embarrassingly long time to look at what I was actually asking these models to do, and it turns out a lot of the tasks had little in common. The solution seemed simple: curate a little stack of go-to models and be intentional about when I'm using which. Qwen 3.5 9B is out, and the 4B is in Dropping down a size gave me a lot more room If you've read any of my local LLM-related articles you probably know by now that Qwen 3.5 9B has been my go-to model since the day I tried it. A colleague recommended it after I was still struggling with gpt-oss 20B. Alibaba's Qwen team released four small Qwen 3.5 models in March, and the 4B and 9B were trained on images and text together, so screenshots work on either one. Going smaller wasn't entirely necessary because the 9B already runs smoothly on my card for the most part, but the thing is, smaller models run faster. The model has to read through all of its weights for every bit of text it generates, and a 4B has less than half as many as the 9B to get through. Context also comes into it. Local models are usually run as compressed 4-bit versions, and at that size the 4B needs roughly 3GB, about half of what the 9B takes. That leaves space to push the context window well past the 30K I usually stick to. These days it gets anything that starts with a screenshot or a quick question I'd rather not send to a cloud model. I also have the 4B and the 2B on my phone, which is where I export my health test results and bank statements straight from their respective mobile apps, so I can query them locally. Before I split up my work between models, this one pretty much got handed everything, from quick questions to coding, design work, research, you name it. Small models do have a habit worth knowing about, though. Artificial Analysis tested the 4B on factual questions and found that when it didn't know an answer, it guessed instead of saying so about 80% of the time. So it's important to give it a web tool. Qwythos took me by surprise My new favorite horsepower model Qwythos 9B v2 is a community fine-tune of that same Qwen 3.5 9B, made by a small lab called Empero AI and retrained to reason through a problem before it answers. I have V2, which came out because the original one had a habit of repeating itself, and Empero's testing put V2's loop rate at zero. I run the 4-bit Q4_K_M file at around 5.7GB, and it fits fully on my RTX 3070. Qwythos has honestly started to replace Gemma 4 for me, which used to be the main model I'd reach for general chat. Something about how it answers reminds me of Claude Sonnet before the 5 models, and it got even stronger once I actually gave it a system prompt which was modeled on how Claude responds. Reasoning is where it pulls ahead of the model it's built on. Empero has it scoring 93.6% on GSM8K, a set of grade-school math word problems, against 80.6% for the base 9B. The jump on GPQA-diamond, which is graduate-level science questions, is bigger, from 32.8% to 49%. Those numbers are self-reported, but they line up with how it handles step-by-step queries. Qwythos now gets my conversations that tend to involve a lot of back and forth especially when it comes to more complex subjects like redesigning a portfolio or putting together a research document. Its weak spots are images, so those still go to Qwen. And code is also not its strong suit. A small coding model for someone who doesn't code Everyone should keep a coding model around I don't write code, which is exactly why I keep models on hand that are tuned for coding, because I clearly need the help. I don't even really build apps, most of my "coding" work is more coding-adjacent, like YAML, JSON, regex, and so on. Though I have also started writing Python scripts lately for a couple of small custom utilities on my PC. By "writing" here I mean asking an AI to write it for me. Scripts I've grabbed off GitHub go to it as well, so it can explain what they'll do to my PC before I run anything. FluentlyQwen3-Coder-4B is a community fine-tune from Project Fluently, built on Qwen3 4B, a generation older than the other two models here, and trained specifically for writing code. Qwen doesn't make an official coding model this small, so a community one is about the only option at 4B. Mine is a 4-bit version on Hugging Face by mradermacher at around 2.4GB, and it only handles text. These jobs used to land on whichever general model happened to be loaded, which wasn't a smart move for me. Anything that needs serious power is always going to go to Claude, but small and personal coding and coding-adjacent jobs go to Fluently now. Giving each model less to do made them better With reasoning and code living with Qwythos and Fluenty, Qwen 3.5 4B ended up with roughly one job, which is research through the Brave Search and fetch tools I added to my runner, and handling my everyday questions. It searches and reads the page before answering, and its reasoning is decent enough. I think the extra context room, due to its small size, matters more than any benchmarks though.