TY - GEN
T1 - Performance of M-ary soft fusion systems using simulated human responses
AU - Bucci, Donald J.
AU - Acharya, Sayandeep
AU - Kam, Moshe
N1 - Publisher Copyright:
© 2014 International Society of Information Fusion.
PY - 2014/10/3
Y1 - 2014/10/3
N2 - A major hurdle in the development of soft and hard/soft data fusion systems is the inability to determine the practical performance gains between fusion operators without the burdens associated with human testing. Drift diffusion models of human responses (i.e., decision, confidence assessments, and response times) from cognitive psychology can be used to gain a sense of the performance of a fusion system during the design phase without the need for human testing. The majority of these models were developed for binary decision tasks, and furthermore, the few models which can operate on M-ary decision tasks are yet unable to generate subject confidence assessments. The current study proposes a method for realizing human responses over an M-ary decision task using pairwise successive comparisons of related binary decision tasks. We provide an example based on the two-stage dynamic signal detection models developed by Pleskac and Busemeyer (2010) where subjects were presented with a pair of lines on a computer screen, asked to determine which of two lines was the longest, and to assess their confidence in their decision using a subjective probability scale. M-ary human opinions were simulated for this line length task and used to assess the performance of several fusion rules, namely: Bayes' rule of probability combination, Dempster's Rule of Combination (DRC), Yager's rule, Dubois and Prade's rule (DPR), and the Proportional Conflict Redistribution rule #5. When taking source reliability into account in the combination, Bayes' rule of probability combination and DRC exhibited the most accurate performance (i.e., the largest amount of specific evidence committed towards the true outcome) for this task. Yager's rule and DPR exhibited inferior performance across all simulated cases.
AB - A major hurdle in the development of soft and hard/soft data fusion systems is the inability to determine the practical performance gains between fusion operators without the burdens associated with human testing. Drift diffusion models of human responses (i.e., decision, confidence assessments, and response times) from cognitive psychology can be used to gain a sense of the performance of a fusion system during the design phase without the need for human testing. The majority of these models were developed for binary decision tasks, and furthermore, the few models which can operate on M-ary decision tasks are yet unable to generate subject confidence assessments. The current study proposes a method for realizing human responses over an M-ary decision task using pairwise successive comparisons of related binary decision tasks. We provide an example based on the two-stage dynamic signal detection models developed by Pleskac and Busemeyer (2010) where subjects were presented with a pair of lines on a computer screen, asked to determine which of two lines was the longest, and to assess their confidence in their decision using a subjective probability scale. M-ary human opinions were simulated for this line length task and used to assess the performance of several fusion rules, namely: Bayes' rule of probability combination, Dempster's Rule of Combination (DRC), Yager's rule, Dubois and Prade's rule (DPR), and the Proportional Conflict Redistribution rule #5. When taking source reliability into account in the combination, Bayes' rule of probability combination and DRC exhibited the most accurate performance (i.e., the largest amount of specific evidence committed towards the true outcome) for this task. Yager's rule and DPR exhibited inferior performance across all simulated cases.
KW - Belief fusion
KW - Data fusion
KW - Dempster-Shafer Theory
KW - Expert reasoning systems
KW - Human Simulation
UR - http://www.scopus.com/inward/record.url?scp=84910595142&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84910595142&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84910595142
T3 - FUSION 2014 - 17th International Conference on Information Fusion
BT - FUSION 2014 - 17th International Conference on Information Fusion
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 17th International Conference on Information Fusion, FUSION 2014
Y2 - 7 July 2014 through 10 July 2014
ER -