Model collapse
The loss of quality and variety when models are trained, generation after generation, on text written by earlier models.
A 2024 paper in Nature by Ilia Shumailov and colleagues showed that when each new model learns mostly from the previous model's output, rare patterns disappear first and the outputs grow narrower and stranger over generations. This raised concern that a web filling up with machine-written text could make future training data worse. Other researchers have found that mixing synthetic data with a steady supply of real data, rather than replacing it, largely avoids the effect. The practical lesson is that the provenance of data matters.