Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
CoRR(2024)
摘要
Self-supervised representation learning has been highly promising for
histopathology image analysis with numerous approaches leveraging their
patient-slide-patch hierarchy to learn better representations. In this paper,
we explore how the combination of domain specific natural language information
with such hierarchical visual representations can benefit rich representation
learning for medical image tasks. Building on automated language description
generation for features visible in histopathology images, we present a novel
language-tied self-supervised learning framework, Hierarchical Language-tied
Self-Supervision (HLSS) for histopathology images. We explore contrastive
objectives and granular language description based text alignment at multiple
hierarchies to inject language modality information into the visual
representations. Our resulting model achieves state-of-the-art performance on
two medical imaging benchmarks, OpenSRH and TCGA datasets. Our framework also
provides better interpretability with our language aligned representation
space. Code is available at https://github.com/Hasindri/HLSS.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要