Layer
One repeated block of a model. Text passes through dozens of layers, each refining the representation a little.
Also: transformer block
A transformer is built from identical layers stacked on top of each other. Each layer takes the hidden state of every token, lets the tokens exchange information through attention, processes each token through a feed-forward network, and passes the result upward, with residual connections and normalisation around both steps. GPT-2 small had 12 layers, GPT-3 had 96, and Llama 3.1 405B has 126. Research suggests lower layers handle spelling and syntax and higher layers handle meaning and the shape of the answer.