Medical Image Synthesis via Fine-Grained Image-Text Alignment and Anatomy-Pathology Prompting
CoRR(2024)
摘要
Data scarcity and privacy concerns limit the availability of high-quality
medical images for public use, which can be mitigated through medical image
synthesis. However, current medical image synthesis methods often struggle to
accurately capture the complexity of detailed anatomical structures and
pathological conditions. To address these challenges, we propose a novel
medical image synthesis model that leverages fine-grained image-text alignment
and anatomy-pathology prompts to generate highly detailed and accurate
synthetic medical images. Our method integrates advanced natural language
processing techniques with image generative modeling, enabling precise
alignment between descriptive text prompts and the synthesized images'
anatomical and pathological details. The proposed approach consists of two key
components: an anatomy-pathology prompting module and a fine-grained
alignment-based synthesis module. The anatomy-pathology prompting module
automatically generates descriptive prompts for high-quality medical images. To
further synthesize high-quality medical images from the generated prompts, the
fine-grained alignment-based synthesis module pre-defines a visual codebook for
the radiology dataset and performs fine-grained alignment between the codebook
and generated prompts to obtain key patches as visual clues, facilitating
accurate image synthesis. We validate the superiority of our method through
experiments on public chest X-ray datasets and demonstrate that our synthetic
images preserve accurate semantic information, making them valuable for various
medical applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要