Pose Priors from Language Models
arxiv(2024)
摘要
We present a zero-shot pose optimization method that enforces accurate
physical contact constraints when estimating the 3D pose of humans. Our central
insight is that since language is often used to describe physical interaction,
large pretrained text-based models can act as priors on pose estimation.
We can thus leverage this insight to improve pose estimation by converting
natural language descriptors, generated by a large multimodal model (LMM), into
tractable losses to constrain the 3D pose optimization. Despite its simplicity,
our method produces surprisingly compelling pose reconstructions of people in
close contact, correctly capturing the semantics of the social and physical
interactions. We demonstrate that our method rivals more complex
state-of-the-art approaches that require expensive human annotation of contact
points and training specialized models. Moreover, unlike previous approaches,
our method provides a unified framework for resolving self-contact and
person-to-person contact.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要