Residual connection
A shortcut that adds a layer's input to its output, so signals and gradients can pass through very deep networks.
Also: skip connection
A residual connection lets each layer add a change to its input rather than replace it. The idea was made famous by the ResNet image model of 2015, which used it to train networks over 100 layers deep. Transformers put residual connections around both the attention and feed-forward steps. Without them, the training signal tends to fade as it travels back through dozens of layers. With them, the hidden state becomes a shared stream that every layer reads from and writes to.