Learning credible deep neural networks with rationale regularization

Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

30 Scopus citations

Abstract

Recent explainability related studies have shown that state-of-the-art DNNs do not always adopt correct evidences to make decisions. It not only hampers their generalization but also makes them less likely to be trusted by end-users. In pursuit of developing more credible DNNs, in this paper we propose CREX, which encourages DNN models to focus more on evidences that actually matter for the task at hand, and to avoid overfitting to data-dependent bias and artifacts. Specifically, CREX regularizes the training process of DNNs with rationales, i.e., a subset of features highlighted by domain experts as justifications for predictions, to enforce DNNs to generate local explanations that conform with expert rationales. Even when rationales are not available, CREX still could be useful by requiring the generated explanations to be sparse. Experimental results on two text classification datasets demonstrate the increased credibility of DNNs trained with CREX. Comprehensive analysis further shows that while CREX does not always improve prediction accuracy on the held-out test set, it significantly increases DNN accuracy on new and previously unseen data beyond test set, highlighting the advantage of the increased credibility.

Original languageEnglish (US)
Title of host publicationProceedings - 19th IEEE International Conference on Data Mining, ICDM 2019
EditorsJianyong Wang, Kyuseok Shim, Xindong Wu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages150-159
Number of pages10
ISBN (Electronic)9781728146034
DOIs
StatePublished - Nov 2019
Externally publishedYes
Event19th IEEE International Conference on Data Mining, ICDM 2019 - Beijing, China
Duration: Nov 8 2019Nov 11 2019

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
Volume2019-November
ISSN (Print)1550-4786

Conference

Conference19th IEEE International Conference on Data Mining, ICDM 2019
Country/TerritoryChina
CityBeijing
Period11/8/1911/11/19

All Science Journal Classification (ASJC) codes

  • General Engineering

Keywords

  • Credibility
  • Deep neural network
  • Expert rationales
  • Explainability

Fingerprint

Dive into the research topics of 'Learning credible deep neural networks with rationale regularization'. Together they form a unique fingerprint.

Cite this