Skip to content

Time to first token

Short answer

Time to first token is the delay between sending a question and the first word of the answer appearing — the pause, as distinct from the typing speed that follows.

It is dominated by how much text the model has to read before it can start writing. Ask about a short page and it is near-instant; ask about a 40-page PDF and the model has to process every selected passage first.

Two numbers describe a local assistant honestly and most products quote only one. A model can have an excellent tokens-per-second figure and still feel slow because every answer begins with a four-second pause.

Related terms

Other ways people ask this

  • what is time to first token

Related