Pre-training
The first and largest stage of training, in which a model learns from trillions of tokens of general text.
Also: pretraining
In pre-training, a model starts from random weights and learns by next-token prediction over a huge corpus of web pages, books, code and other text. It is by far the most expensive stage: frontier models use thousands to tens of thousands of graphics processors running for months. The result is a base model that can continue any text but does not yet reliably follow instructions or hold a conversation. Later stages, collectively called post-training, shape that raw ability into a useful and safer assistant.