My 8GB GPU is now running a 27B model, and it's doing the work I used to pay ChatGPT for

My 8GB GPU is now running a 27B model, and it's doing the work I used to pay ChatGPT for

Published Nov 17, 2026, 4:00 PM EST Nolen began their writing career in 2019, with three years dedicated to editing the Creative section at MakeUseOf. Their expertise lies at the crossroads of technology and creativity, covering areas like photography, video editing, and graphic design. Outside of work, you'll often find Nolen diving into a good book, writing their own stories, or playing video games. The monthly $20 subscription for ChatGPT covers more than just the chatbox these days. Parts of it include your saved setups for the same jobs that need repeating, adjusting a model's reasoning effort, dropping in your files for synthesis, and of course, access to better models. In contrast, local models usually get sold as the private version of just the chat box, while the rest get left out. An 8GB graphics card doesn't look like the hardware to change that, either. The models that fit on one tend to be small and not very capable of replacing a whole workflow the way ChatGPT can handle it. But I may just have found one that comes close. The 27B model that runs on an 8GB card It doesn't need a 27B budget to work well PrismML, a startup founded by Caltech researchers, released Bonsai 27B in July. It's a compressed version of Qwen 3.6 27B, one of Alibaba's open models, and the 1-bit build I use stores each of the model's weights, the numbers it's made of, in about 1.125 bits instead of 16. That makes it roughly 14 times smaller than the full model, and that's pretty much the only reason I'm able to run it on my small card at all. PrismML builds the compression into training instead of squeezing a finished file down afterwards. I reckon that's why it doesn't behave like most heavily squashed models I've tried. The whole model actually fits on my GPU with some room left to spare, so I can push up the context to 30-40k. Swapping custom GPTs for presets ChatGPT's saved setups, but make it local Custom GPTs are ChatGPT's version of a saved setup; they contain instructions and reference files you can open whenever a specific job keeps coming back around. OpenAI will be retiring them on December 11, and Plus users already can't build new ones, so Projects are the closest equivalent left on that plan. There is a local workaround to this, and it actually has more to do with the runner or workspace you run Bonsai in than the model itself, at least for the initial setup. LM Studio calls them presets, and each one saves a system prompt, the standing instructions a model reads before every chat, along with its sampling settings. These are actually parts that ChatGPT never let you touch. Temperature decides how adventurous each word choice gets. Min P throws out any word that's far less likely than the model's top pick, which helps a warmer preset stay coherent, and OpenAI doesn't offer it even to developers using its API. My brainstorming preset runs warm with Min P switched on at 0.05. Meanwhile, my presets that are built around strict rules get a much colder temperature, and that's where Bonsai follows its instructions most closely. A warm preset with Min P off starts letting the rules slip a few replies in. Presets don't hold reference files the way a GPT did, so documents still have to be attached to sessions separately, which is a little extra admin, but so is everything with local AI. Thinking mode versus ChatGPT's effort slider Not every question needs it to think first ChatGPT Plus has a Thinking slider in the chat box with Instant, Medium, and High, which sets how long the model works through a question before answering. Bonsai thinks by default, but this isn't necessarily a good thing depending on the task, which is why I usually turn it off for most tasks. Thinking eats into my 30K of context and slows every reply, and ChatGPT dealt with that on OpenAI's servers, so I never had to think about it before. Multi-step problems are what I switch it on for most of the time, for example, comparing two phone contracts over 24 months once the promo price runs out. PrismML scored all its benchmarks with thinking on, so the quality they advertise assumes it's switched on. How much it loses with it off, I couldn't tell you, but it seems that math is the first to go. With thinking on, Bonsai is at its best on anything that has a right answer but takes a few steps to reach, like math or a set of rules. ChatGPT on High would likely still beat it on a hard problem. But where Bonsai wins is that the reasoning is all there to read in your runner, while OpenAI also doesn't publish how many Thinking messages Plus gets before it moves you to another model. Handing Bonsai my documents and screenshots My files stay on my PC Throwing a file at ChatGPT is about as easy as it gets, and Plus allows up to 80 uploads every three hours. LM Studio takes .docx, .pdf, and .txt files dropped straight into a chat. A file that fits in the context goes in whole, while the stuff that's longer gets searched instead, and only the parts that look relevant to the question reach the model, which is what happens to most of my longer files at 30K context. Documents are where Bonsai does well. Qwen3.6, the model it's built on, was designed with long files in mind, and every fourth layer uses regular attention to pick out exact details, so dates and amounts buried halfway down a page don't get lost. A clause in a lease or a letter from my bank can get explained to me in plain terms, and it keeps track of which section said what better than my 9B models. Images are a little weaker for Bonsai. The vision part of the model is a separate 4-bit piece that loads alongside it, so it's been compressed hard too. It reads the text in a screenshot fine, so it's not terrible, but it can get a little loose with smaller details. I don't need OpenAI anymore Custom GPTs have less than a month left, so the presets on my PC will outlive them, which is a little strange considering all of this is running on a graphics card I bought for gaming years ago. It's not like I'm abandoning ChatGPT, but with Bonsai capable of handling most of its core functions, I don't see the point in paying for it anymore.

Original Source

Read the full article at Xda-developers →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.