Can you run Qwen 3.5 2B on Apple M4?
Short answer
Qwen 3.5 2B will run on the CPU alone, and whether that is usable depends on the model.
- Model file
- 1.2 GB
- Needs in VRAM, with context
- about 2.3 GB
- Memory
- system RAM, shared
- Measured on this hardware
- 15 tokens/sec
How this was worked out
Qwen 3.5 2B is a 1.2 GB file. Running it needs the weights in memory plus room for the context window and the runtime, which comes to roughly 2.3 GB in practice. Apple M4 has no dedicated video memory, so the model runs in system RAM on the CPU. That is arithmetic rather than a benchmark: it tells you whether the model loads, not how it feels.
What we have actually measured
On RTX 3060 12 GB, Vulkan backend this model generated 15 tokens per second, against 15 with no graphics card at all. That is one machine, and yours will differ.
If it is tight
There is no card to run out of, so the limit is patience rather than memory. Small models are fine here; the large ones are not. Nightjar sizes the context window to the memory it finds, so a smaller card gets a shorter window rather than a failed load.
Smaller models that fit the same card
- Llama 3.2 1B on Apple M4 — 0.75 GB
Related
- Up a levelCan your PC run a local AI model?
- Closely relatedCan you run Gemma 4 12B-it on Apple M4?
- Closely relatedCan you run Gemma 4 26B-A4B-it on Apple M4?
- Closely relatedCan you run Gemma 4 31B-it on Apple M4?
- Closely relatedCan you run Gemma 4 E2B-it on Apple M4?
- Elsewhere on the siteLocal AI on Apple M4: which models run and how fast
- Elsewhere on the siteQwen 3.5 2B in Nightjar: speed, size and what it is good at