A published reading test for every bundled model
Short answer
Each model in the catalogue is run against a fixed 16-question set covering page comprehension, table lookup and refusal to answer from nothing, and the scores are published.
The problem it exists for
Parameter counts do not tell you whether a small model can read a table. Without a test, choosing between eleven models is guesswork dressed up as preference.
How it works
- The same 16 questions are asked of every model, with known correct answers.
- Scores are published alongside file sizes and measured speed.
- The results are not what size alone would predict: Qwen 3.5 2B scored 16/16 while the much larger Qwen 3.5 9B scored 14.
What it does not do
- Sixteen questions is a small set. It catches a model that cannot read a table and cannot separate two models that both can.
- It measures document reading, which is this product's job, and says nothing about general capability.
- One run on one machine, with no variance reported because there is not enough data to report it honestly.
Other ways people ask this
- browser model reading test