I turned on one Gemma 4 feature most people skip, and my local AI got 137% faster

Published Oct 30, 2026, 5:30 PM EDT Abhinav pivoted from a career in banking to pursue his first love in writing. Even while working full-time, he continued contributing as an editor-at-large, a role he has held for more than 7 years. A lifelong tech enthusiast who has built three gaming and productivity powerhouse PCs since 2018, his passion for technology keeps him closely following the semiconductor industry, from NVIDIA and AMD to ARM. His MSc dissertation explored how artificial intelligence will reshape the future of work, reflecting his curiosity about the wider social impact of emerging technologies. The local AI landscape is moving at a breakneck pace. As a matter of fact, it's moving ahead so quickly that anyone on a llama.cpp build from just a couple of months ago could already be behind. New releases continue to bring new features that can change what your hardware is capable of, and sometimes, the gap more often than not means the difference between a model that's usable and a model that's capable. If you don't believe me, you need to look no further than the example of the recent developments that have come to Google's family of open-weight models, Gemma 4. Earlier in May, Google shipped multi-token prediction drafters for Gemma models, with llama.cpp adding support for it in June. The drafter happens to be a 242-megabyte file that, on my RTX 4070 Ti Super, boosted Gemma 4 12B from about 75 tokens per second to an average of 179 tokens per second, which is a gain of a whopping 137% across three workloads that I tested myself. Here's how you, too, can make use of it. My Gemma 4 12B model was fast, but it could be much faster 75 tokens/second was the ceiling, until it wasn't To establish how fast a model can get, the first step, of course, is to measure how fast it currently runs. I tested my model's baseline speed on my PC with an RTX 4070 Ti Super with 16GB of GDDR6X memory coupled with a Ryzen 5 7600X and 32GB of DDR5 memory first. I served Unsloth's Gemma 4 12B QAT build at the UD-Q4_K_XL quant through llama-server on llama.cpp build b11927, with 32K context, flash attention, every layer on the GPU, and Google's recommended sampler settings (temperature 1.0, top-p 0.95, top-k 64). In all my tests, I turned the thinking mode off, so that every generated token was counted as a part of an answer. I gave the model three jobs, including generating a JSON list of fictional PC hardware components, a Python duplicate-file finder, and a short story of 600 words. I gave it the following three prompts in a fresh chat. Generate a JSON array of 15 fictional PC components. Each object must have these fields: id, name, category, brand, price_usd, release_year, and specs (an object with three fields relevant to the category). Output only the JSON, no commentary. Write a complete Python command-line tool that recursively scans a directory, finds duplicate files by SHA-256 hash, and prints a report grouped by hash showing each file's path and size. Use argparse, handle permission errors gracefully, and include docstrings. Output only the code. Write a short story of about 600 words about a lighthouse keeper who slowly realizes the light has been signaling to something out at sea, and that something has started signaling back. As you can tell, each of these prompts test a unique use-case and represents the prompts users generally send to a model of this size and capability tier. After running each prompt, I recorded the following results. The test methodology comes from the llama.cpp pull request that added Gemma 4 MTP, wherein another tester on the same GPU reported the results in the three categories I've used for the test. Task (Baseline) Run 1 Run 2 Run 3 Average of 3 runs (tokens/sec) JSON 77.1 74.4 72.8 74.7 Code 74.3 74.9 77.2 75.5 Prose 73.7 77.0 76.8 75.8 Switching on multi-token prediction took one file and four flags No new GPU and no smaller models, just a drafter Google made for the purpose The way that multi-token prediction (MTP) works is by pairing the target model (here, Gemma 4 12B) with a small drafter model that was trained alongside it. Google details this in their documentation, explaining that MTP is the architecture Gemma 4 uses for speculative decoding. Speculative decoding is essentially a way to get more out of each pass through a large model by letting the small drafter model do the guesswork for token prediction, while the large target model confirms several of those guesses in the time it would normally take to output a single token. In this case, it means that, rather than having the target model generate every token one at a time, the drafter predicts several tokens ahead, and then, the large model checks all of them in parallel. When a "guess" is rejected by the target model, it still produces the correct token for that position, so the step isn't wasted, and the drafter picks up from there. For this test, I downloaded both files from Unsloth's Gemma 4 12B QAT repository on Hugging Face, including the UD-Q4_K_XL build and its matching drafter, mtp-gemma-4-12B-it.gguf, which was a conversion of Google's very own drafter. It was important to keep the pair from one repo, because a QAT model should run with its QAT drafter. To switch MTP on, I had to restart llama-server with four additional flags, with everything else from the baseline run kept identical. I then used four flags: -md mtp-gemma-4-12B-it.gguf (to load the drafter), -ngld 99 (to put the drafter on the GPU alongside the main model), --spec-type draft-mtp (to run the drafter as a Gemma 4 MTP head that reads the main model's cache, rather than a standalone model) and --spec-draft-n-max 3 (which caps each round at three guesses, which is a value I took from similar results on the same GPU in the llama.cpp pull request). Gemma 4 got faster at everything I asked it to write Code and JSON nearly tripled, and even prose got 73% faster With the drafter loaded, the only thing left was to run the test again under identical VRAM conditions. There's no other way to put this, besides simply stating that every task got faster. The JSON and code tasks jumped from about 75 tokens per second to around 200, as brackets, field names, and boilerplate are all easy to predict and complemented the drafter-equipped model. What came as a surprise, however, was the fact that even the prose workload, which gives the drafter more room to miss, ended up running 73% faster. Task MTP off MTP on Gain JSON 74.7 199.4 +167% Code 75.5 205.4 +172% Prose 75.8 131.3 +73% Average 75.3 178.7 +137% llama.cpp Grab your drafter before you upgrade your GPU If you run Gemma 4 in llama.cpp, getting a drafter model seems to be one of the easiest and most straightforward ways to get a speed boost for it. All that you need to do is update to the latest build of llama.cpp, download the drafter that matches your model, and add four flags. The only thing that's worth noting is that dense models like the Gemma 4 12B model I tested benefit from a drafter the most. While I haven't tested any mixture-of-experts models with a drafter yet, Google says that those models benefit less from a drafter on single-user setups. It's also worth noting that the results also vary depending on the specific hardware configuration that you're on, but testers have reported gains on everything from an RTX 5090 to a GT 1030, so there's nothing to lose from giving it a go.