Performance of probability transformations using simulated human opinions

Donald J. Bucci, Sayandeep Acharya, Timothy J. Pleskac, Moshe Kam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Probability transformations provide a method of relating Dempster-Shafer sources of evidence to subjective probability assignments. These transforms are constructed to facilitate decision making over a set of mutually exclusive hypotheses. The probability information content (PIC) metric has been recently proposed for characterizing the performance of different probability transforms. To investigate the applicability of the PIC metric, we compare five probability transformations (i.e., BetP, PrPl, PrNPl, PrHyb, and DSmP) using a simulator of human responses from cognitive psychology known as two-stage dynamic signal detection. Responses were simulated over two tasks: a line length discrimination task and a city population size discrimination task. Human decision-makers were modeled for these two tasks by Pleskac and Busemeyer (2010). Subject decisions and confidence assessments were simulated and combined for both tasks using Yager's rule and mapped into subjective probabilities using the five probability transforms. Receiver operating characteristic (ROC) curves, normalized areas under the ROC curves (AUCs), along with average PIC values were obtained for each probability transform. Our results indicate that higher PIC values do not necessarily equate to higher discriminability (i.e., higher normalized AUCs) between probability transforms. In fact, all five probability transforms exhibited nearly the same normalized AUC values. At lower, fixed false alarm rates, the BetP, PrPl, PrNPl, and PrHyb transforms yielded higher detection rates over the DSmP transform. For higher, fixed false alarm rates, the DSmP transform yielded higher detection rates over the other four transforms. These trends were observed over both tasks, which suggests that the PIC may not be sufficient for evaluating the performance of probability transforms.

Original languageEnglish (US)
Title of host publicationFUSION 2014 - 17th International Conference on Information Fusion
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9788490123553
StatePublished - Oct 3 2014
Externally publishedYes
Event17th International Conference on Information Fusion, FUSION 2014 - Salamanca, Spain
Duration: Jul 7 2014Jul 10 2014

Publication series

NameFUSION 2014 - 17th International Conference on Information Fusion

Other

Other17th International Conference on Information Fusion, FUSION 2014
Country/TerritorySpain
CitySalamanca
Period7/7/147/10/14

All Science Journal Classification (ASJC) codes

  • Information Systems

Keywords

  • Belief fusion
  • Data fusion
  • Dempster-Shafer theory
  • Human Simulation
  • Probability transformations

Fingerprint

Dive into the research topics of 'Performance of probability transformations using simulated human opinions'. Together they form a unique fingerprint.

Cite this