Skip to content

Chunking

Short answer

Chunking is splitting a long document into passages small enough to retrieve and fit in a context window.

Where the splits fall matters more than it sounds. Cut mid-table and the model sees headers without rows; cut mid-clause and it sees an obligation without its exception.

Chunk size is a trade: small chunks retrieve precisely but lose surrounding context, large chunks keep context but crowd out other passages. Most visible failures of document assistants are chunking failures rather than model failures.

Related terms

Other ways people ask this

  • what is chunking

Related