Separating the Wheat from the Chaff: Comparative Visual Cues for Transparent Diagnostics of Competing Models

Aritra Dasgupta, Hong Wang, Nancy O'Brien, Susannah Burrows

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


Experts in data and physical sciences have to regularly grapple with the problem of competing models. Be it analytical or physics-based models, a cross-cutting challenge for experts is to reliably diagnose which model outcomes appropriately predict or simulate real-world phenomena. Expert judgment involves reconciling information across many, and often, conflicting criteria that describe the quality of model outcomes. In this paper, through a design study with climate scientists, we develop a deeper understanding of the problem and solution space of model diagnostics, resulting in the following contributions: i) a problem and task characterization using which we map experts' model diagnostics goals to multi-way visual comparison tasks, ii) a design space of comparative visual cues for letting experts quickly understand the degree of disagreement among competing models and gauge the degree of stability of model outputs with respect to alternative criteria, and iii) design and evaluation of MyriadCues, an interactive visualization interface for exploring alternative hypotheses and insights about good and bad models by leveraging comparative visual cues. We present case studies and subjective feedback by experts, which validate how MyriadCues enables more transparent model diagnostic mechanisms, as compared to the state of the art.

Original languageEnglish (US)
Article number8812989
Pages (from-to)1043-1053
Number of pages11
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number1
StatePublished - Jan 2020

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


  • Model evaluation
  • Simulation
  • Transparency
  • Visual comparison
  • Visual cues


Dive into the research topics of 'Separating the Wheat from the Chaff: Comparative Visual Cues for Transparent Diagnostics of Competing Models'. Together they form a unique fingerprint.

Cite this