Local AI on RTX 3090: which models run and how fast
Short answer
RTX 3090 has 24 GB of video memory, which holds 10 of Nightjar's eleven models entirely, the largest being Qwen 3.5 35B-A3B.
- Video memory
- 24 GB
- Models that fit fully
- 10 of 11
- Largest that fits
- Qwen 3.5 35B-A3B
- This is our test card
- no
Every model against this card
- Llama 3.2 1B — 0.75 GB, fits with 22.2 GB spare
- Qwen 3.5 2B — 1.2 GB, fits with 21.7 GB spare
- Qwen 3.5 4B — 2.6 GB, fits with 20.3 GB spare
- Gemma 4 E2B-it — 3.2 GB, fits with 19.7 GB spare
- Gemma 4 E4B-it — 5 GB, fits with 17.9 GB spare
- Qwen 3.5 9B — 5.3 GB, fits with 17.6 GB spare
- Gemma 4 12B-it — 6.6 GB, fits with 16.3 GB spare
- Gemma 4 26B-A4B-it — 15.8 GB, fits with 7.1 GB spare
- Qwen 3.5 27B — 15.6 GB, fits with 7.3 GB spare
- Qwen 3.5 35B-A3B — 20.5 GB, fits with 2.4 GB spare
- Gemma 4 31B-it — 30.4 GB, needs CPU offload
What the numbers here are and are not
The fit is arithmetic — a file size against a memory size, with about 1.1 GB added for context and runtime. The speeds elsewhere on this site were measured on RTX 3060 12 GB, Vulkan backend and nowhere else. Where a page says a figure was not measured, that is literal: we would rather leave a gap than publish an estimate that reads like a result.
Other ways people ask this
- best local llm for rtx 3090
- rtx 3090 llm tokens per second
- I have a RTX 3090. Which local model should I run for browser AI?