Self-attention
Attention in which the tokens of one text attend to each other, rather than to a separate input.
In self-attention, the queries, keys and values all come from the same sequence, so every token can look at every other token in that sequence. This is what lets a transformer build an understanding of a whole passage. Its cost grows with the square of the sequence length: twice as many tokens means about four times the work, which is why very long context windows are expensive. In decoder models, self-attention is masked so that each token can see only the tokens before it.