I tested every local LLM tweak for real work, and the best ones contradict cloud LLM advice

Published Oct 30, 2026, 2:01 PM EDT Nolen began their writing career in 2019, with three years dedicated to editing the Creative section at MakeUseOf. Their expertise lies at the crossroads of technology and creativity, covering areas like photography, video editing, and graphic design. Outside of work, you'll often find Nolen diving into a good book, writing their own stories, or playing video games. A lot of the advice for getting better results from AI models comes from the companies running those models in data centers. Their documentation is pretty detailed and actually sensible for the most part, the issue is that a lot of local LLM guides end up repeating that same advice. But local AI on an 8GB GPU can't be tuned or used the same as a hefty Claude model on Anthropic's servers. After having run local LLMs for real work for the better part of this year, I realized the tweaks and tricks often go against the grain of what cloud providers recommend. Keeping "Thinking" turned off The feature cloud models brag about is the one I turn off first Thinking, or reasoning mode, is when a model writes out its work before giving you an answer. The idea is that it's meant to be more accurate on complex tasks. Most of my models have a thinking mode, including my Qwen, Gemma 4, and Qwythos models. Cloud providers lean hard on the thinking thing, to the point where some won't even let you opt out. Claude Opus 5.5 runs with thinking permanently on, and Google says every Gemini 3.x model always thinks too, so the closest you get is turning the effort down. OpenAI does let developers set GPT-5.5's reasoning effort to none, but the default is medium, and OpenAI only suggests dropping it if you want a quick reply more than a clever one. I keep Thinking off on pretty much all my local models, and context is the main reason. My runner's context window has to hold everything, meaning my prompt, the conversation history, any files I added, and also the model's thinking. Keeping it turned off is basically like a budgeting hack for a bit of extra savings. It legitimately saves me a few hundred up to a thousand tokens per prompt. Any prompt that really needed thinking that badly was going to go to my cloud models anyway because that's still what I reach for with complex or multi-step tasks. Switching on Min P sampling A setting cloud users have never even been offered Min P is a sampling setting, so it changes how the model picks each next word. It sets a minimum cutoff that scales with how likely the model's top choice is, then throws out the long tail of unlikely words underneath it. This is an excellent setting to turn up when you have a high temperature (more random than analytical) because it basically prevents the model from going a little off the rails. Whenever my temp is over 0.3, I bump up Min P between 0.03 and 0.05. Cloud tools don't offer it at all - OpenAI, Anthropic, and Google don't expose Min P as a setting, and the bigger trend is that they don't expose these sampling controls at all. Claude Sonnet 5 rejects any non-default temperature or top P value, and Anthropic tells developers to use prompting instead. OpenAI and Google aren't much different. GPT-5.5 only accepts the default temperature, and while Google still lets you change it on Gemini 3, it strongly recommends sticking to 1.0 because lower values can cause looping. For what it's worth, Qwen's own recommended settings leave Min P at 0.0, but I still keep it on whenever I have a higher temperature because it prevents replies from becoming unusable. Capping the context window Cloud models brag about millions of tokens, I keep mine very low Context length is how many tokens a model can hold at once. Like I've already mentioned, everything counts towards it, including your prompts, files, and the model's responses. Cloud models have pushed that number past a million, with GPT-5.5 sitting just above 1M and Claude Sonnet 5 serving a 1M window by default. Anthropic even publishes tips specifically for individual prompts carrying 20k tokens of documents. What's interesting is that Anthropic's own context engineering guide warns about context rot, which is when recall gets worse when the window starts filling up. This happens a lot sooner with local models if you don't have a massive card, which is why I keep my context around 30k usually. I can get it running smoothly on up to 80k, but I need to account for other apps I'm running at the same time as well, which I usually am, especially when a local model is plugged into a design tool or agentic workspace. My models technically go way higher - Qwen 3.5 9B supports 262k natively, but that's just not an option for my GPU. A lower context window is actually not that limiting. It forces you to really think through your prompts and be selective with what to add to a session. It can get a bit restrictive when I'm trying to do multi-step tasks though. Reaching for a smaller model first The advice to start big doesn't matter when your GPU is small Cloud providers usually tell you to start big. OpenAI's model selection guide says to begin with the most capable model and only test smaller ones once cost and latency become more relevant. And Anthropic says most workloads should start with Claude Opus 5.5. I mean, this is probably mainly marketing - of course every AI company wants you to try their most capable models so the cheaper ones end up feeling like a downgrade. When it comes to local AI, I start small intentionally; usually Gemma E2B, E4B, or Qwen 3.5 4B. It simply comes down to conserving system resources - if those models can handle the task fine, why even bother scaling up? Qwen 3.5 4B gets all my tiny coding jobs like fixing little buttons, and my Gemma models get my quick chats. My little GPU plays by different rules Cloud guides are written with data centers in mind, and they don't really account for a model squeezed onto an 8GB card that also needs to run a notes app, a browser, and an image editor. I think the more useful habit is treating every default like a suggestion, then poking at it on your own hardware and deciding what actually works for your setup.