Differentiated Explanation of Deep Neural Networks With Skewed Distributions

Weijie Fu, Meng Wang, Mengnan Du, Ninghao Liu, Shijie Hao, Xia Hu

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Over the last decade, deep neural networks (DNNs) are regarded as black-box methods, and their decisions are criticized for the lack of explainability. Existing attempts based on local explanations offer each input a visual saliency map, where the supporting features that contribute to the decision are emphasized with high relevance scores. In this paper, we improve the saliency map based on differentiated explanations, of which the saliency map not only distinguishes the supporting features from backgrounds but also shows the different degrees of importance of the various parts within the supporting features. To do this, we propose to learn a differentiated relevance estimator called DRE, where a carefully-designed distribution controller is introduced to guide the relevance scores towards right-skewed distributions. DRE can be directly optimized under pure classification losses, enabling higher faithfulness of explanations and avoiding non-trivial hyper-parameter tuning. The experimental results on three real-world datasets demonstrate that our differentiated explanations significantly improve the faithfulness with high explainability. Our code and trained models are available at https://github.com/fuweijie/DRE.

Original languageEnglish (US)
Pages (from-to)2909-2922
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume44
Issue number6
DOIs
StatePublished - Jun 1 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Keywords

  • Deep neural networks
  • differentiated saliency maps
  • local explanation
  • relevance scores

Fingerprint

Dive into the research topics of 'Differentiated Explanation of Deep Neural Networks With Skewed Distributions'. Together they form a unique fingerprint.

Cite this