IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models
arxiv(2023)
摘要
We present a diffusion-based image morphing approach with
perceptually-uniform sampling (IMPUS) that produces smooth, direct and
realistic interpolations given an image pair. The embeddings of two images may
lie on distinct conditioned distributions of a latent diffusion model,
especially when they have significant semantic difference. To bridge this gap,
we interpolate in the locally linear and continuous text embedding space and
Gaussian latent space. We first optimize the endpoint text embeddings and then
map the images to the latent space using a probability flow ODE. Unlike
existing work that takes an indirect morphing path, we show that the model
adaptation yields a direct path and suppresses ghosting artifacts in the
interpolated images. To achieve this, we propose a heuristic bottleneck
constraint based on a novel relative perceptual path diversity score that
automatically controls the bottleneck size and balances the diversity along the
path with its directness. We also propose a perceptually-uniform sampling
technique that enables visually smooth changes between the interpolated images.
Extensive experiments validate that our IMPUS can achieve smooth, direct, and
realistic image morphing and is adaptable to several other generative tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要