Frontier quality now runs on a 16GB gaming GPU

Last week, in the Qwen3.8-Max write-up, I said the question this tracker exists to answer is: when will frontier-quality answers run on hardware a small business can own? I expected Qwen3.8's promised open weights to answer it. They still haven't landed.

A different model answered it first.

Qwen3.6-35B-A3B, running 4-bit on my 16GB RTX 5070 Ti, lands at +0.005 [−0.01, +0.02] against my frozen frontier line. Statistically at the frontier, tied with Qwen3.8-Max for the best point estimate in the series, ahead of every other open model I've tested, and the first to get there running locally. It runs at 66 tokens per second, uses 10.9GB of VRAM, and handles images. The full run cost $1.24, all of it judge fees. The model itself cost nothing to run.

The usual disclosure, briefly

My benchmark is ~163 tasks of realistic everyday work: coding, repo-level fixes, document extraction, long-context retrieval, structured output, reasoning, and hard judgment calls. Every model is scored against four frozen frontier anchors, with a judge panel doing the first pass and hand-audits on top. The tasks stay private so they stay out of training runs. The methodology summary and full standings are on the tracker page. What you get here is scores, deltas, and failure stories.

How this run happened

This wasn't a release-day intake. I was picking a daily driver: one local multimodal model to use for everything. Four candidates fit a 16GB card on paper:

The winner posted perfect 1.00s on four dimensions: reasoning, factuality, long context, and instruction following. On identical content, indistinguishable from the frontier anchors. The weak spots are small: repo-level coding 0.94, image work 0.95, isolated coding 0.96. It was also the cleanest run I've recorded: no truncation artifacts, no harness surprises, nothing that needed re-inspection.

Gemma 4 12B is a solid second: 0.925, faster than the winner, and the only candidate that accepts audio input. If you want a smaller daily driver, pick that.

Why the bigger model fits and the smaller one doesn't

The odd result first: the 35B model runs comfortably on a 16GB card, and its 27B dense sibling doesn't fit at all. The dense model's 4-bit weights plus vision encoder need 16.4GB. I have 16.3GB available, before any KV cache. Partially offloaded, it ran at 12 tokens per second with the GPU 83% idle, waiting on PCIe. I stopped the run after an hour of a projected twenty-four.

The MoE is larger on paper but activates only 3B parameters per token. llama.cpp keeps the always-used layers on the GPU and parks the experts in system RAM: 66 tokens per second, 5.6x faster than the "smaller" model. On a 16GB card, mixture-of-experts is not a compromise. It's the architecture that fits.

What the losers taught me about harnesses

GLM-4.6V-Flash first scored 0.762. Its real score, after two fixes with identical weights, is 0.801. Both fixes matter if you run local models:

• Runaway thinking starves the answer. On one spreadsheet task it produced 55,000 characters of reasoning, hit the output ceiling, and returned an empty answer. It scored zero on a task it never got to attempt. llama.cpp's --reasoning-budget cap took that task from 0.00 to 1.00. A control re-run showed the cap doesn't inflate other scores, it only prevents this failure.
• Language drift. It answered one English prompt entirely in Chinese, correctly, and my scoring patterns expect English. A one-line system prompt fixed it. Model error, not scorer error, but the lesson is the same: what looks like a bad model is often a bad harness setup.

Its final 0.801 still carries a warning. It scored a perfect 1.00 on image tasks while scoring 0.62 on factuality and 0.62 on coding. Image understanding has saturated: all four candidates and every frontier anchor score near the top. A dimension everyone aces can't pick your winner anymore, and a small model's best number can hide its real profile.

I also owe the tracker a correction. Gemma 4 12B's published 0.80 was wrong, and the mistake was mine. In July its default thinking mode rambled so long that answers never finished, so I ran it with thinking off as a workaround. The workaround became the measurement: the same weights with thinking on (and capped) score 0.925, with reasoning going from 0.39 to 1.00. A setting I chose to make the model runnable changed what I was measuring, and the wrong number stayed published for three weeks. The tracker page carries the correction as of today.

Serving notes, if you deploy it

• Thinking is ON by default. That's the opposite of Qwen3.8-Max three weeks ago, and the fifth vendor default-flip I've logged. Always check.
• Cap the reasoning budget. Uncapped, some tasks spend the entire output allowance on thinking and return nothing.
• The config that carried the run: llama.cpp, experts offloaded to CPU (--n-cpu-moe), larger prefill batch, 4-bit UD-Q4_K_XL weights. First token typically arrives inside two seconds. Decode holds ~66 t/s.

Caveats, honestly

• Private suite, single run. Read tiers, not decimals. The confidence interval spans zero, so "at frontier" means indistinguishable from the anchors, not ahead of them.
• I measured the 4-bit quantization, because that's what fits the hardware this question is about. Full-precision numbers would differ, presumably upward.
• My repo tasks are single-shot. Long-horizon agentic coding, the thing every vendor now markets, is exactly what this suite doesn't measure yet. That dimension is under construction.
• Image tasks have saturated suite-wide, so "handles images" here means "keeps up where everyone keeps up."

What happens next

Qwen3.8's open weights are still promised and still absent. The day they land, the 27B goes through the full suite and this post gets a sequel. Until then, the tracker's founding question has an answer: frontier-level quality, within noise, now runs on a 16GB GPU in an ordinary desktop. Full standings and the changelog are on Vikas's Open Model Benchmark.