Lost in the middle
The tendency of models to use information at the start and end of a long input better than information in the middle.
A 2023 study by Nelson Liu and colleagues at Stanford found that when the answer to a question sat in the middle of many retrieved documents, models answered less accurately than when it sat at the beginning or end, a U-shaped curve similar to human memory. The finding shaped how developers order documents in retrieval-augmented systems. Newer long-context models show the effect less strongly, but it remains a reason to put the most important material near the start or end of a prompt.