Skip to content

Benchmarks

Every speed figure on this site came from one machine — RTX 3060 12 GB, Vulkan backend. That is a limitation, not a flourish, and it is why pages about other hardware say “not measured” instead of showing a number.

Generation speed

ModelFileGPU, tok/sCPU, tok/sEval
Llama 3.2 1B0.75 GB——10/16
Qwen 3.5 2B1.2 GB651516/16
Qwen 3.5 4B2.6 GB——15/16
Gemma 4 E2B-it3.2 GB691314/16
Gemma 4 E4B-it5 GB——13/16
Qwen 3.5 9B5.3 GB273.214/16
Gemma 4 12B-it6.6 GB242.816/16
Gemma 4 26B-A4B-it15.8 GB———
Qwen 3.5 27B15.6 GB———
Qwen 3.5 35B-A3B20.5 GB———
Gemma 4 31B-it30.4 GB———

4 of 11 models have been timed. A dash means we have not run it, not that it is slow. The evaluation column is a 16-question set covering reading comprehension over a page, table lookup and refusal to answer from nothing.

Memory, 50 tabs

  • All 50 live6,253 MB
  • 49 frozen1,070 MB
  • 49 unloaded633 MB

Load time, filtering on

  • Yahoo9,633 ms → 1,093 ms88.7%
  • Euronews29.8%
  • CNNwithin noise6.6%
  • Wikipediacontrol — little to block2%
  • Hacker Newscontrol — little to block4.8%

Method

Generation speed is tokens per second on a sustained response, taken after the model is loaded and warm, with the Vulkan backend. The CPU column is the same prompt with no GPU offload at all. The tab figures are resident memory with 50 tabs open, measured in three states: everything live, 49 frozen, and 49 unloaded.

What is wrong with these numbers

One machine. No variance reported, because there are not enough runs to report variance honestly. The blocking figures predate the current filtering engine and have not been rerun, which is why Yahoo’s 88.7% should be read as “large” rather than as a current measurement. Wikipedia and Hacker News are in that table as controls: 2.0% and 4.8% is roughly the noise floor, so CNN’s 6.6% is not a result.

Run it yourself

The models are ordinary GGUF files and the backend is open. If your figures differ from ours, yours are the ones that describe your machine. Start with your graphics card for the arithmetic on what will fit before you measure what it does, and see Vulkan against CUDA for why the default backend is the slower one on NVIDIA hardware.