Consensus-based evaluation framework for distributed information retrieval systems

Authors:
Jason J. Jung
Affiliations:
Yeungnam University, Department of Computer Engineering, Dae-Dong, Gyeonsan, South Korea
Venue:
Knowledge and Information Systems
Year:
2009

Citing 0
Cited 7

Contextualized query sampling to discover semantic resource descriptions on the web

Information Processing and Management: an International Journal
An evolutionary approach to query-sampling for heterogeneous systems

Expert Systems with Applications: An International Journal
Semantic preprocessing for mining sensor streams from heterogeneous environments

Expert Systems with Applications: An International Journal
Exploiting multi-agent platform for indirect alignment between multilingual ontologies: A case study on tourism business

Expert Systems with Applications: An International Journal
Attribute selection-based recommendation framework for long-tail user group: an empirical study on movieLens dataset

ICCCI'11 Proceedings of the Third international conference on Computational collective intelligence: technologies and applications - Volume Part I
Evolutionary approach for semantic-based query sampling in large-scale information sources

Information Sciences: an International Journal
Attribute selection-based recommendation framework for short-head user group: An empirical study by MovieLens and IMDB

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.02

Visualization

Abstract

Multi-agent systems have been attacking the challenges of information retrieval tasks on distributed environment. In this paper, we propose a consensus choice selection method based framework to evaluate the performance of cooperative information retrieval tasks of the multiple agents. Thereby, two well-known measurements, precision and recall, are extended to handle consensual closeness (i.e., local and global consensus) between the sets of retrieved results. We show that in a motivating example the proposed criteria are prone to solve the rigidity problem of classical precision and recall. More importantly, the retrieved results can be ranked with respect to the consensual score, and the ranking mechanism has been verified to be more reasonable.