Out of memory when loading a model
Short answer
The model plus its context needs roughly its file size plus 1.1 GB. If that exceeds your VRAM or free RAM, it will not load — use a smaller model.
Why this happens
Weights are only part of the requirement. The KV cache for the context window grows alongside them, which is where the extra gigabyte goes, and it grows further as a conversation gets long.
What to do
- 01Work out the requirement: model file size + about 1.1 GB. A 5.3 GB model wants roughly 6.4 GB available.
- 02Compare that with your VRAM, or with free system RAM if you have no discrete card.
- 03Pick a model that fits with headroom rather than one that just fits. A model that loads and then fails mid-conversation is worse than one that was never chosen.
- 04If a model loaded before and now does not, start a new conversation. A long history enlarges the cache.
Other ways people ask this
- nightjar out of memory when loading a model
- Nightjar Browser: out of memory when loading a model. How do I fix it?