C-Eval: A Unified Metric to Evaluate Feature-based Explanations via Perturbation

Minh N. Vu, Truc D. Nguyen, Nhat Hai Phan, Ralucca Gera, My T. Thai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In many image-classification applications, understanding the reasons of model's prediction can be as critical as the prediction's accuracy itself. Various feature-based local explainers have been designed to provide explanations on the decision of complex classifiers. Nevertheless, there is no consensus on evaluating the quality of different explanations. In response to this lack of comprehensive evaluation, we introduce the c-Eval metric and its corresponding framework to quantify the feature-based local explanation's quality. Given a classifier's prediction and the corresponding explanation on that prediction, c-Eval is the minimum-distortion perturbation that successfully alters the prediction while keeping the explanation's features unchanged. To show that c-Eval captures the importance of input's features, we establish a connection between c-Eval and the features returned by explainers in affine and nearly-affine classifiers. We then introduce the c-Eval plot, which not only displays a strong connection between c-Eval and explainers' quality, but also helps automatically determine explainer's parameters.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021
EditorsYixin Chen, Heiko Ludwig, Yicheng Tu, Usama Fayyad, Xingquan Zhu, Xiaohua Tony Hu, Suren Byna, Xiong Liu, Jianping Zhang, Shirui Pan, Vagelis Papalexakis, Jianwu Wang, Alfredo Cuzzocrea, Carlos Ordonez
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages927-937
Number of pages11
ISBN (Electronic)9781665439022
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Big Data, Big Data 2021 - Virtual, Online, United States
Duration: Dec 15 2021Dec 18 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021

Conference

Conference2021 IEEE International Conference on Big Data, Big Data 2021
Country/TerritoryUnited States
CityVirtual, Online
Period12/15/2112/18/21

All Science Journal Classification (ASJC) codes

  • Information Systems and Management
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Information Systems

Keywords

  • Explainable/Interpretable Machine Learning
  • Feature-based Local Explainers
  • Image Classification
  • Metric

Fingerprint

Dive into the research topics of 'C-Eval: A Unified Metric to Evaluate Feature-based Explanations via Perturbation'. Together they form a unique fingerprint.

Cite this