DiffusionGPT: LLM-Driven Text-to-Image Generation System
CoRR(2024)
摘要
Diffusion models have opened up new avenues for the field of image
generation, resulting in the proliferation of high-quality models shared on
open-source platforms. However, a major challenge persists in current
text-to-image systems are often unable to handle diverse inputs, or are limited
to single model results. Current unified attempts often fall into two
orthogonal aspects: i) parse Diverse Prompts in input stage; ii) activate
expert model to output. To combine the best of both worlds, we propose
DiffusionGPT, which leverages Large Language Models (LLM) to offer a unified
generation system capable of seamlessly accommodating various types of prompts
and integrating domain-expert models. DiffusionGPT constructs domain-specific
Trees for various generative models based on prior knowledge. When provided
with an input, the LLM parses the prompt and employs the Trees-of-Thought to
guide the selection of an appropriate model, thereby relaxing input constraints
and ensuring exceptional performance across diverse domains. Moreover, we
introduce Advantage Databases, where the Tree-of-Thought is enriched with human
feedback, aligning the model selection process with human preferences. Through
extensive experiments and comparisons, we demonstrate the effectiveness of
DiffusionGPT, showcasing its potential for pushing the boundaries of image
synthesis in diverse domains.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要