Training compute
The total amount of computation used to train a model, usually counted in floating-point operations.
Also: FLOPs, compute budget
Training compute counts the arithmetic needed to train a model, measured in floating-point operations, or FLOPs. A handy rule is about six operations per parameter per training token, so GPT-3, with 175 billion parameters and 300 billion tokens, needed about 3 times 10 to the 23 FLOPs. Frontier models in 2024 and 2025 are estimated at 10 to the 25 or more. Compute thresholds appear in regulation: the EU AI Act presumes systemic risk for general-purpose models trained with more than 10 to the 25 FLOPs.