FlashAttention
A way of computing exact attention that avoids slow trips to GPU memory, making long contexts faster and cheaper.
Also: flash attention
FlashAttention, published by Tri Dao and colleagues at Stanford in 2022, does not change what attention computes, only how. It splits the work into tiles that fit in a graphics processor's small, fast on-chip memory and never writes the full table of attention scores to the slower main memory. The result is the same answer, several times faster and with memory that grows in line with sequence length rather than its square. Later versions tuned it for newer chips, and it is now standard in training and inference software.