Local AI on CPU only: which models run and how fast
Short answer
CPU only has no dedicated video memory, so models run on the CPU in system RAM. The small models are usable there; the large ones are not.
- Video memory
- none, shared with system RAM
- Models that fit fully
- 0 of 11
- Largest that fits
- none — CPU only
- This is our test card
- no
Every model against this card
- Llama 3.2 1B — 0.75 GB, CPU only
- Qwen 3.5 2B — 1.2 GB, CPU only
- Qwen 3.5 4B — 2.6 GB, CPU only
- Gemma 4 E2B-it — 3.2 GB, CPU only
- Gemma 4 E4B-it — 5 GB, CPU only
- Qwen 3.5 9B — 5.3 GB, CPU only
- Gemma 4 12B-it — 6.6 GB, CPU only
- Gemma 4 26B-A4B-it — 15.8 GB, CPU only
- Qwen 3.5 27B — 15.6 GB, CPU only
- Qwen 3.5 35B-A3B — 20.5 GB, CPU only
- Gemma 4 31B-it — 30.4 GB, CPU only
What the numbers here are and are not
The fit is arithmetic — a file size against a memory size, with about 1.1 GB added for context and runtime. The speeds elsewhere on this site were measured on RTX 3060 12 GB, Vulkan backend and nowhere else. Where a page says a figure was not measured, that is literal: we would rather leave a gap than publish an estimate that reads like a result.
Other ways people ask this
- best local llm for cpu only
- cpu only llm tokens per second
- I have a CPU only. Which local model should I run for browser AI?