Skip to content

Context overflow

Short answer

Context overflow is what happens when the prompt, the retrieved passages and the conversation together exceed the model's window — the oldest material is dropped.

The failure is quiet. The model does not announce that it has forgotten the start of the conversation; it simply answers as though it never happened, which reads like carelessness rather than a limit.

Starting a new conversation for a new document is the simple fix, and on a long document it is worth asking narrower questions so retrieval has less to fit.

Related terms

Other ways people ask this

  • what is context overflow

Related