Automated Radiological Impression Generation For Plain Chest X-Rays With End To End Deep Learning

The chest X-Ray (CXR) is the one of the most common clinical exam used to diagnose thoracic diseases and abnormalities. The volume of CXR scans generated daily in hospitals is huge. Therefore, an automated diagnosis system that is able to save the effort of doctors is of great value. At present, the applications of artificial intelligence in CXR diagnosis usually use pattern recognition to classify the scans. However, such methods rely on labeled databases. They are costly and usually have a high error rate. In this work, we built a database containing more than 12,000 CXR scans and radiological reports, and developed a model based on deep convolutional neural network and recurrent network with attention mechanism. The model learns features from the CXR scans and the associated raw radiological reports directly; no additional labeling required. The model provides automated recognition of given scans and generation of impression. The quality of the generated impression was evaluated with both the CIDEr scores and by radiologists as well. The CIDEr scores were found to be around 5.8 on average for the testing dataset. Further blind evaluation suggested a comparable performance against radiologists.