Cost-benefit methodology for office systems
ACM Transactions on Information Systems (TOIS)
Cost benefit analysis of information systems: a survey of methodologies
COCS '88 Proceedings of the ACM SIGOIS and IEEECS TC-OA 1988 conference on Office information systems
Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval
Information-seeking and mediated searching. Part 1: theoretical framework and research design
Journal of the American Society for Information Science and Technology
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
Optimizing search engines using clickthrough data
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
A survey on the use of relevance feedback for information access systems
The Knowledge Engineering Review
A community-aware search engine
Proceedings of the 13th international conference on World Wide Web
Adaptive web search based on user profile constructed without any effort from users
Proceedings of the 13th international conference on World Wide Web
Exploiting Query Repetition and Regularity in an Adaptive Community-Based Web Search Engine
User Modeling and User-Adapted Interaction
Context-sensitive information retrieval using implicit feedback
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
The Turn: Integration of Information Seeking and Retrieval in Context (The Information Retrieval Series)
A study on the effects of personalization and task information on implicit feedback performance
CIKM '06 Proceedings of the 15th ACM international conference on Information and knowledge management
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
ACM Transactions on Information Systems (TOIS)
An eye tracking study of the effect of target rank on web search
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
An investigation on a community's web search variability
ACSC '08 Proceedings of the thirty-first Australasian conference on Computer science - Volume 74
A cost-benefit evaluation method for web-based information systems
Proceedings of the 20th ACM conference on Hypertext and hypermedia
Choices in batch information retrieval evaluation
Proceedings of the 18th Australasian Document Computing Symposium
| Hi-index | 0.00 |
The utility of an enterprise search system is determined by three key players: the information retrieval (IR) system (the search engine), the enterprise users, and the service provider who delivers the tailored IR service to its designated enterprise users. Currently, evaluations of enterprise search have been focused largely on the IR system effectiveness and efficiency, only a relatively small amount of effort on the user's involvement, and hardly any effort on the service provider's role. This paper will investigate the role of the service provider. We propose a method that evaluates the cost and benefit for a service provider of using a mediated search engine - in particular, where domain experts intervene on the ranking of the search results from a search engine. We test our cost and benefit evaluation method in a case study and conduct user experiments to demonstrate it. Our study shows that: 1) by making use of domain experts' relevance assessments in search result ranking, the precision and the discount cumulated gain of ranked lists have been improved significantly (144% and 40% respectively); 2) the service provider gains substantial return on investment and higher search success rate by investing in domain experts' relevance assessments; and 3) the cost and benefit evaluation also indicates the type of queries to be selected from a query log for evaluating an enterprise search engine.