Can you run Gemma 4 12B-it on Apple M2?
Short answer
Gemma 4 12B-it will run on the CPU alone, and whether that is usable depends on the model.
- Model file
- 6.6 GB
- Needs in VRAM, with context
- about 7.7 GB
- Memory
- system RAM, shared
- Measured on this hardware
- 2.8 tokens/sec
How this was worked out
Gemma 4 12B-it is a 6.6 GB file. Running it needs the weights in memory plus room for the context window and the runtime, which comes to roughly 7.7 GB in practice. Apple M2 has no dedicated video memory, so the model runs in system RAM on the CPU. That is arithmetic rather than a benchmark: it tells you whether the model loads, not how it feels.
What we have actually measured
On RTX 3060 12 GB, Vulkan backend this model generated 2.8 tokens per second, against 2.8 with no graphics card at all. That is one machine, and yours will differ.
If it is tight
There is no card to run out of, so the limit is patience rather than memory. Small models are fine here; the large ones are not. Nightjar sizes the context window to the memory it finds, so a smaller card gets a shorter window rather than a failed load.
Smaller models that fit the same card
- Gemma 4 E2B-it on Apple M2 — 3.2 GB
- Gemma 4 E4B-it on Apple M2 — 5 GB
- Qwen 3.5 9B on Apple M2 — 5.3 GB
Related
- Up a levelCan your PC run a local AI model?
- Closely relatedCan you run Gemma 4 12B-it on Intel Arc A770?
- Closely relatedCan you run Gemma 4 12B-it on Intel Arc B580?
- Closely relatedCan you run Gemma 4 12B-it on CPU only?
- Closely relatedCan you run Gemma 4 12B-it on GTX 1660 Ti?
- Elsewhere on the siteLocal AI on Apple M2: which models run and how fast
- Elsewhere on the siteGemma 4 12B-it in Nightjar: speed, size and what it is good at