Concordance and consensus

  • Authors:

  • Cees Elzinga;Hui Wang;Zhiwei Lin;Yash Kumar

  • Affiliations:

  • Department of Sociology, PARIS Research Program, VU University Amsterdam, The Netherlands;Computer Science Research Institute, School of Computing and Mathematics, University of Ulster, Northern Ireland, UK;Computer Science Research Institute, School of Computing and Mathematics, University of Ulster, Northern Ireland, UK;Computer Science Department, International Institute of Information Technology, Hyderabad, India

  • Venue:
  • Information Sciences: an International Journal
  • Year:
  • 2011

Quantified Score

Hi-index 0.07

Visualization

Abstract

This paper deals with the measurement of concordance and the construction of consensus in preference data, either in the form of preference rankings or in the form of response distributions with Likert-items. We propose a set of axioms of concordance in preference orderings and a new class of concordance measures. The measures outperform classic measures like Kendall's @t and W and Spearman's @r in sensitivity and apply to large sets of orderings instead of just to pairs of orderings. For sets of N orderings of n items, we present very efficient and flexible algorithms that have a time complexity of only O(Nn^2). Remarkably, the algorithms also allow for fast calculation of all longest common subsequences of the full set of orderings. We experimentally demonstrate the performance of the algorithms. A new and simple measure for assessing concordance on Likert-items is proposed.