FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
CoRR(2024)
摘要
As machine learning models in critical fields increasingly grapple with
multimodal data, they face the dual challenges of handling a wide array of
modalities, often incomplete due to missing elements, and the temporal
irregularity and sparsity of collected samples. Successfully leveraging this
complex data, while overcoming the scarcity of high-quality training samples,
is key to improving these models' predictive performance. We introduce
“FuseMoE”, a mixture-of-experts framework incorporated with an innovative
gating function. Designed to integrate a diverse number of modalities, FuseMoE
is effective in managing scenarios with missing modalities and irregularly
sampled data trajectories. Theoretically, our unique gating function
contributes to enhanced convergence rates, leading to better performance in
multiple downstream tasks. The practical utility of FuseMoE in real world is
validated by a challenging set of clinical risk prediction tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要