Time to first token
Short answer
Time to first token is the delay between sending a question and the first word of the answer appearing — the pause, as distinct from the typing speed that follows.
It is dominated by how much text the model has to read before it can start writing. Ask about a short page and it is near-instant; ask about a 40-page PDF and the model has to process every selected passage first.
Two numbers describe a local assistant honestly and most products quote only one. A model can have an excellent tokens-per-second figure and still feel slow because every answer begins with a four-second pause.
Related terms
Other ways people ask this
- what is time to first token