Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding
CoRR(2023)
摘要
In recent years, the results of view-based 3D shape recognition methods have
saturated, and models with excellent performance cannot be deployed on
memory-limited devices due to their huge size of parameters. To address this
problem, we introduce a compression method based on knowledge distillation for
this field, which largely reduces the number of parameters while preserving
model performance as much as possible. Specifically, to enhance the
capabilities of smaller models, we design a high-performing large model called
Group Multi-view Vision Transformer (GMViT). In GMViT, the view-level ViT first
establishes relationships between view-level features. Additionally, to capture
deeper features, we employ the grouping module to enhance view-level features
into group-level features. Finally, the group-level ViT aggregates group-level
features into complete, well-formed 3D shape descriptors. Notably, in both
ViTs, we introduce spatial encoding of camera coordinates as innovative
position embeddings. Furthermore, we propose two compressed versions based on
GMViT, namely GMViT-simple and GMViT-mini. To enhance the training
effectiveness of the small models, we introduce a knowledge distillation method
throughout the GMViT process, where the key outputs of each GMViT component
serve as distillation targets. Extensive experiments demonstrate the efficacy
of the proposed method. The large model GMViT achieves excellent 3D
classification and retrieval results on the benchmark datasets ModelNet,
ShapeNetCore55, and MCB. The smaller models, GMViT-simple and GMViT-mini,
reduce the parameter size by 8 and 17.6 times, respectively, and improve shape
recognition speed by 1.5 times on average, while preserving at least 90
classification and retrieval performance.
更多查看译文
关键词
3D object recognition,Multi-view ViT,View grouping,3D position embedding,Knowledge distillation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要