Feed-forward network
The part of each transformer layer that processes every token on its own, holding much of the model's stored knowledge.
Also: FFN, MLP block
After attention mixes information between tokens, each layer passes every token separately through a small two-layer network, usually widening the vector about four times and then shrinking it back. These feed-forward blocks hold around two-thirds of a dense model's parameters, and studies suggest they store many of its facts. Modern models use gated variants such as SwiGLU. In a mixture-of-experts model, the single feed-forward block is replaced by many expert blocks, of which only a few run for each token.