Heat Death of Generative Models in Closed-Loop Learning
arxiv(2024)
摘要
Improvement and adoption of generative machine learning models is rapidly
accelerating, as exemplified by the popularity of LLMs (Large Language Models)
for text, and diffusion models for image generation.As generative models become
widespread, data they generate is incorporated into shared content through the
public web. This opens the question of what happens when data generated by a
model is fed back to the model in subsequent training campaigns. This is a
question about the stability of the training process, whether the distribution
of publicly accessible content, which we refer to as "knowledge", remains
stable or collapses.
Small scale empirical experiments reported in the literature show that this
closed-loop training process is prone to degenerating. Models may start
producing gibberish data, or sample from only a small subset of the desired
data distribution (a phenomenon referred to as mode collapse). So far there has
been only limited theoretical understanding of this process, in part due to the
complexity of the deep networks underlying these generative models.
The aim of this paper is to provide insights into this process (that we refer
to as "generative closed-loop learning") by studying the learning dynamics of
generative models that are fed back their own produced content in addition to
their original training dataset. The sampling of many of these models can be
controlled via a "temperature" parameter. Using dynamical systems tools, we
show that, unless a sufficient amount of external data is introduced at each
iteration, any non-trivial temperature leads the model to asymptotically
degenerate. In fact, either the generative distribution collapses to a small
set of outputs, or becomes uniform over a large set of outputs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要