Abstract
Existing methods to measure the rank accuracy of a recommender system assume the ground truth is either a set of user ratings or a total ordered list of items given by the user with possible ties. However, in many applications we are only able to obtain implicit user feedback, which does not provide such comprehensive information, but only gives a set of pairwise preferences among items. Generally such pairwise preferences are not complete, and thus may not deduce a total order of items. In this paper, we propose a novel method to evaluate rank accuracy, expected discounted rank correlation, which addresses the unique challenges of handling incomplete pairwise preferences in ground truth and also puts an emphasis on properly ranking items that users most prefer.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 74-77 |
| Number of pages | 4 |
| Journal | CEUR Workshop Proceedings |
| Volume | 811 |
| State | Published - 2011 |
| Externally published | Yes |
| Event | Joint Workshop on Human Decision Making in Recommender Systems, Decisions@RecSys 2011 and User-Centric Evaluation of Recommender Systems and Their Interfaces-2, UCERSTI 2 - Affiliated with the 5th ACM Conference on Recommender Systems, RecSys 2011 - Chicago, IL, United States Duration: Oct 23 2011 → Oct 26 2011 |
All Science Journal Classification (ASJC) codes
- General Computer Science
Keywords
- Evaluation
- Pairwise preference
- Rank accuracy
- Recommender systems