Joint Learning for Pneumonia Classification and Segmentation on Medical Images

Shaobo Liu, Xin Zhong, Frank Y. Shih

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Chest X-ray images are notoriously difficult to analyze due to the noisy nature. Automatic identification of pneumonia on medical images has attracted intensive study recently. In this paper, a novel joint-Task architecture that can learn pneumonia classification and segmentation simultaneously is presented. Two modules, including an image preprocessing module and an attention module, are developed to improve both the classification and segmentation accuracies. Results from the experiments performed on the massive dataset of the Radiology Society of North America have confirmed its superiority over the other existing methods. The classification test accuracy is improved from 0.89 to 0.95, and the segmentation model achieves an improved mean precision result of 0.58-0.78. Finally, two weakly supervised learning methods, class-saliency map and Grad-CAM, are used to highlight the corresponding pixels or areas which have significant influence on the classification model, such that the refined segmentation can focus on the correct areas with high confidence.

Original languageEnglish (US)
Article number2157003
JournalInternational Journal of Pattern Recognition and Artificial Intelligence
Issue number5
StatePublished - Apr 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


  • Deep learning
  • VGG16
  • morphological neural networks
  • pneumonia classification
  • segmentation


Dive into the research topics of 'Joint Learning for Pneumonia Classification and Segmentation on Medical Images'. Together they form a unique fingerprint.

Cite this