Graphics processing unit
The chip that trains and runs language models, built for doing many simple calculations at once.
Also: GPU, accelerator, TPU
Graphics processing units were made for video games but suit neural networks, which are mostly large matrix multiplications. Nvidia's chips, from the A100 to the H100 and Blackwell generations, dominate model training, while Google trains Gemini on its own tensor processing units, or TPUs. A frontier training run uses tens of thousands of such chips linked by fast networks. Access to them is a strategic question: the United States restricts exports of advanced chips to China, and India's IndiaAI Mission subsidises GPU access for Indian model builders.