An efficient but effective writer: Diffusion-based semi-autoregressive transformer for automated radiology report generation

It is firmly believed that manually diagnosing radiology images is clinically critical but labour-intensive and error-prone. Therefore, an automatic radiology report generation method is highly desired for alleviating the burden imposed on doctors. However, a typical report contains numerous template descriptions and only a few abnormal sentences. This unbalanced distribution makes the generation of template sentences more likely. Additionally, describing an entire report in a word-by-word manner can lead to significant latency during the inference step. Besides, the existing datasets are limited to conventional pneumonia, making them incomplete and one-sided. This work is concerned with forming a better trade-off between generation performance. One key design is an abnormal semantic diffusion module, which progressively absorbs the semantics of abnormal medical terminology and strengthens the linguistic coherence between local tokens. In detail, the generated report is refined by enhancing the incorporation of informative words with limited occurrence frequencies, which alleviates the monotony of template-based generation. Another design is a length-controllable self attention decoder, which regulates the input length of the sentences used for target word generation. This framework preserves the autoregressive nature of word generation while also maintaining a controllable range, ensuring the efficiency of report generation. Moreover, a novel XRG-COVID-19 clinical dataset is tailored, which includes X-ray scans and professional diagnostic reports of 8676 patients. The experimental results demonstrate the proposed model achieves a better trade-off between performance and speed than those of carefully designed baselines on both the IU X-ray dataset and the proposed XRG-COVID-19 dataset.