Hidden state
The vector a model holds for each token between layers, carrying everything it has worked out so far.
Also: activation, residual stream
As text moves through a model, each token is represented by a vector the same size as its embedding, often 4,096 to over 16,000 numbers. Each layer reads this hidden state and adds its own contribution back into it, which is why researchers call the running total the residual stream. The final hidden state of the last token is turned into scores for the next token. Interpretability research reads and edits hidden states to find where a model stores ideas such as a place, a language or a refusal.