Skip to content

Replies are too slow

Short answer

Either the model is too large for your graphics card and is partly running on the CPU, or there is no GPU being used at all. Check which model you are on and drop to a smaller one.

Why this happens

When a model does not fit entirely in VRAM, some layers run on the CPU and every token crosses the bus twice. The speed difference is not subtle: on our test machine Qwen 3.5 9B managed 27 tokens per second on the GPU and 3.2 on the CPU.

What to do

  1. 01Check your card against the model on the can-i-run pages. If the fit is tight or partial, that is your answer.
  2. 02Drop one model down. Qwen 3.5 2B is 1.2 GB, scored full marks on our evaluation, and ran at 65 tokens per second on the test card.
  3. 03Close other GPU users — a game, a video call, a second browser playing video all take VRAM.
  4. 04Check the GPU is detected at all. If Nightjar reports CPU-only on a machine with a graphics card, see the GPU not detected page.

Other ways people ask this

  • nightjar replies are too slow
  • Nightjar Browser: replies are too slow. How do I fix it?

Related