MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shading
CoRR(2023)
摘要
Reconstructing an avatar from a portrait image has many applications in
multimedia, but remains a challenging research problem. Extracting reflectance
maps and geometry from one image is ill-posed: recovering geometry is a
one-to-many mapping problem and reflectance and light are difficult to
disentangle. Accurate geometry and reflectance can be captured under the
controlled conditions of a light stage, but it is costly to acquire large
datasets in this fashion. Moreover, training solely with this type of data
leads to poor generalization with in-the-wild images. This motivates the
introduction of MoSAR, a method for 3D avatar generation from monocular images.
We propose a semi-supervised training scheme that improves generalization by
learning from both light stage and in-the-wild datasets. This is achieved using
a novel differentiable shading formulation. We show that our approach
effectively disentangles the intrinsic face parameters, producing relightable
avatars. As a result, MoSAR estimates a richer set of skin reflectance maps,
and generates more realistic avatars than existing state-of-the-art methods. We
also introduce a new dataset, named FFHQ-UV-Intrinsics, the first public
dataset providing intrisic face attributes at scale (diffuse, specular, ambient
occlusion and translucency maps) for a total of 10k subjects. The project
website and the dataset are available on the following link:
https://ubisoftlaforge.github.io/character/mosar
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要