Emergent abilities
Skills that seem to appear suddenly once a model passes a certain size, and the debate over whether they are real.
Also: emergence
In 2022 Jason Wei and colleagues described abilities, such as multi-step arithmetic, that were near zero in smaller models and rose sharply in larger ones, calling them emergent. The idea suggested that scaling could bring surprising new capabilities. In 2023 Rylan Schaeffer and colleagues at Stanford argued that much of the sudden jump came from all-or-nothing scoring, and that with smoother measures the improvement looked gradual. Both views agree that larger models can do more; they differ on whether the gains are predictable.