AI trained on AI garbage spits out AI garbage

As AI-generated content proliferates online, the quality of AI models trained on this data is at risk of degradation, according to new research published in Nature. The study, led by Ilia Shumailov from the University of Oxford, highlights a phenomenon known as "model collapse," where models produce incoherent outputs due to training on data generated by other AI models. This process is likened to repeatedly photographing a photograph, leading to a loss of clarity and detail. The research indicates that as AI models increasingly rely on junk content from the internet, their performance may decline, with slower improvements and higher perplexity scores indicating less accurate predictions.

The study involved fine-tuning a large language model (LLM) on Wikipedia data and then on its own outputs over nine generations, revealing that the quality of generated text deteriorated significantly. Experts emphasize the importance of high-quality, diverse training data, noting that reliance on synthetic data could exacerbate issues, particularly for underrepresented languages. To mitigate degradation, one suggestion is to ensure models retain access to original human-generated data. However, distinguishing between human and AI-generated content remains a challenge. The findings underscore the need for careful consideration of data provenance to maintain the integrity and reliability of AI models in the face of increasing AI-generated noise on the internet.