Variations in relevance judgments and the measurement of retrieval effectiveness
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
The use of MMR, diversity-based reranking for reordering documents and producing summaries
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval
A probabilistic model of information retrieval: development and comparative experiments
Information Processing and Management: an International Journal
An information-theoretic approach to automatic query expansion
ACM Transactions on Information Systems (TOIS)
SIGIR '02 Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Cumulated gain-based evaluation of IR techniques
ACM Transactions on Information Systems (TOIS)
ACM SIGIR Forum
Beyond independent relevance: methods and evaluation metrics for subtopic retrieval
Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Retrieval evaluation with incomplete information
Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval
Accurately interpreting clickthrough data as implicit feedback
Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
Redundant documents and search effectiveness
Proceedings of the 14th ACM international conference on Information and knowledge management
Learning to rank using gradient descent
ICML '05 Proceedings of the 22nd international conference on Machine learning
Building a reusable test collection for question answering
Journal of the American Society for Information Science and Technology - Research Articles
A risk minimization framework for information retrieval
Information Processing and Management: an International Journal - Special issue: Formal methods for information retrieval
Learning user interaction models for predicting web search result preferences
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Improving web search ranking by incorporating user behavior information
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Less is more: probabilistic models for retrieving fewer relevant documents
SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Information Systems
Reliable information retrieval evaluation with incomplete and biased judgements
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
A support vector method for optimizing average precision
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
The relationship between IR effectiveness measures and user satisfaction
SIGIR '07 Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval
Novelty and topicality in interactive information retrieval
Journal of the American Society for Information Science and Technology
Ambiguous queries: test collections need more sense
Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval
Proceedings of the Second ACM International Conference on Web Search and Data Mining
Enhancing diversity, coverage and balance for summarization through structure learning
Proceedings of the 18th international conference on World wide web
An axiomatic approach for result diversification
Proceedings of the 18th international conference on World wide web
What Else Is There? Search Diversity Examined
ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
Portfolio theory of information retrieval
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Multiple approaches to analysing query diversity
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Modeling Expected Utility of Multi-session Information Distillation
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
An Effectiveness Measure for Ambiguous and Underspecified Queries
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
An Analysis of NP-Completeness in Novelty and Diversity Ranking
ICTIR '09 Proceedings of the 2nd International Conference on Theory of Information Retrieval: Advances in Information Retrieval Theory
Effective Keyword Search for Software Resources Installed in Large-Scale Grid Infrastructures
WI-IAT '09 Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 01
Interactive, topic-based visual text summarization and analysis
Proceedings of the 18th ACM conference on Information and knowledge management
Expected reciprocal rank for graded relevance
Proceedings of the 18th ACM conference on Information and knowledge management
Probabilistic models of ranking novel documents for faceted topic retrieval
Proceedings of the 18th ACM conference on Information and knowledge management
Jointly optimising relevance and diversity in image retrieval
Proceedings of the ACM International Conference on Image and Video Retrieval
Redundancy, diversity and interdependent document relevance
ACM SIGIR Forum
Clustering for music search results
ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Actively predicting diverse search intent from user browsing behaviors
Proceedings of the 19th international conference on World wide web
Diversifying web search results
Proceedings of the 19th international conference on World wide web
Exploiting query reformulations for web search result diversification
Proceedings of the 19th international conference on World wide web
Proceedings of the 19th international conference on World wide web
Increasing precision and diversity in photo retrieval by result fusion
CLEF'08 Proceedings of the 9th Cross-language evaluation forum conference on Evaluating systems for multilingual and multimodal information access
DivQ: diversification for keyword search over structured databases
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Mining the blogosphere for top news stories identification
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Studying trailfinding algorithms for enhanced web search
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Do user preferences and evaluation measures line up?
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Assessing the scenic route: measuring the value of search trails in web logs
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Estimating interference in the QPRP for subtopic retrieval
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Metrics for assessing sets of subtopics
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Diversification of search results using webgraphs
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Supervised query modeling using wikipedia
Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
DivRank: the interplay of prestige and diversity in information networks
Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
ACM SIGIR Forum
Wikipedia as sense inventory to improve diversity in Web search results
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
ACM SIGMOD Record
What can quantum theory bring to information retrieval
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Learning to rank relevant and novel documents through user feedback
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
A framework for evaluating database keyword search strategies
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Selectively diversifying web search results
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Exploiting novelty, coverage and balance for topic-focused multi-document summarization
CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Approximation algorithms for diversified search ranking
ICALP'10 Proceedings of the 37th international colloquium conference on Automata, languages and programming: Part II
Mining and explaining relationships in wikipedia
DEXA'10 Proceedings of the 21st international conference on Database and expert systems applications: Part II
On the evaluation of entity profiles
CLEF'10 Proceedings of the 2010 international conference on Multilingual and multimodal information access evaluation: cross-language evaluation forum
A comparative analysis of cascade measures for novelty and diversity
Proceedings of the fourth ACM international conference on Web search and data mining
Proceedings of the fourth ACM international conference on Web search and data mining
Multi-dimensional search result diversification
Proceedings of the fourth ACM international conference on Web search and data mining
Towards a collection-based results diversification
RIAO '10 Adaptivity, Personalization and Fusion of Heterogeneous Information
A weighted-graph-based approach for diversifying search results
International Journal of Knowledge and Web Intelligence
An analysis of NP-completeness in novelty and diversity ranking
Information Retrieval
A unified framework for recommending diverse and relevant queries
Proceedings of the 20th international conference on World wide web
Search result diversity for informational queries
Proceedings of the 20th international conference on World wide web
On the informativeness of cascade and intent-aware effectiveness measures
Proceedings of the 20th international conference on World wide web
Efficient diversification of web search results
Proceedings of the VLDB Endowment
Efficient diversity-aware search
Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Summarization of personal photologs using multidimensional content and context
Proceedings of the 1st ACM International Conference on Multimedia Retrieval
Identifying relevant social media content: leveraging information diversity and user cognition
Proceedings of the 22nd ACM conference on Hypertext and hypermedia
Find it if you can: a game for modeling different types of web search success using interaction data
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Incremental diversification for very large sets: a streaming-based approach
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Intent-aware search result diversification
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
System effectiveness, user models, and user utility: a conceptual framework for investigation
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Evaluating diversified search results using per-intent graded relevance
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Evaluating multi-query sessions
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
How diverse are web search results?
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Intent-oriented diversity in recommender systems
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Multidimensional search result diversification: diverse search results for diverse users
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
To diversify or not to diversify entity summaries on RDF knowledge graphs?
ISMIS'11 Proceedings of the 19th international conference on Foundations of intelligent systems
Model-based inference about IR systems
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
Aggregated search result diversification
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
Do subtopic judgments reflect diversity?
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
A query-basis approach to parametrizing novelty-biased cumulative gain
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
Using emotion to diversify document rankings
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
A query performance analysis for result diversification
ICTIR'11 Proceedings of the Third international conference on Advances in information retrieval theory
Searching the wikipedia with public online search engines
INEX'10 Proceedings of the 9th international conference on Initiative for the evaluation of XML retrieval: comparative evaluation of focused retrieval
Rank and relevance in novelty and diversity metrics for recommender systems
Proceedings of the fifth ACM conference on Recommender systems
Workshop on novelty and diversity in recommender systems - DiveRS 2011
Proceedings of the fifth ACM conference on Recommender systems
Sparse spatial selection for novelty-based search result diversification
SPIRE'11 Proceedings of the 18th international conference on String processing and information retrieval
Diversifying Product Review Rankings: Getting the Full Picture
WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Interest and Evaluation of Aggregated Search
WI-IAT '11 Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01
Diversifying search results of controversial queries
Proceedings of the 20th ACM international conference on Information and knowledge management
Suggestion set utility maximization using session logs
Proceedings of the 20th ACM international conference on Information and knowledge management
Coreference aware web object retrieval
Proceedings of the 20th ACM international conference on Information and knowledge management
Simulating simple user behavior for system effectiveness evaluation
Proceedings of the 20th ACM international conference on Information and knowledge management
Diversification for multi-domain result sets
Proceedings of the 20th ACM international conference on Information and knowledge management
Recency ranking by diversification of result set
Proceedings of the 20th ACM international conference on Information and knowledge management
Diverse retrieval via greedy optimization of expected 1-call@k in a latent subtopic relevance model
Proceedings of the 20th ACM international conference on Information and knowledge management
Evaluating large-scale distributed vertical search
Proceedings of the 9th workshop on Large-scale and distributed informational retrieval
Intent-based diversification of web search results: metrics and algorithms
Information Retrieval
TIARA: Interactive, Topic-Based Visual Text Summarization and Analysis
ACM Transactions on Intelligent Systems and Technology (TIST)
Large-scale validation and analysis of interleaved search evaluation
ACM Transactions on Information Systems (TOIS)
IR system evaluation using nugget-based test collections
Proceedings of the fifth ACM international conference on Web search and data mining
Effective query formulation with multiple information sources
Proceedings of the fifth ACM international conference on Web search and data mining
2nd international workshop on diversity in document retrieval (DDR 2012)
Proceedings of the fifth ACM international conference on Web search and data mining
Explicit search result diversification through sub-queries
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Evaluation and user preference study on spatial diversity
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Using the quantum probability ranking principle to rank interdependent documents
ECIR'2010 Proceedings of the 32nd European conference on Advances in Information Retrieval
Ranking with submodular valuations
Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms
Evaluating subtopic retrieval methods: Clustering versus diversification of search results
Information Processing and Management: an International Journal
A non-parametric visual-sense model of images--extending the cluster hypothesis beyond text
Multimedia Tools and Applications
CICLing'10 Proceedings of the 11th international conference on Computational Linguistics and Intelligent Text Processing
Answer diversification for complex question answering on the web
PAKDD'10 Proceedings of the 14th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part I
Introducing diversity to log-based query suggestions to deal with underspecified user queries
SIIS'11 Proceedings of the 2011 international conference on Security and Intelligent Information Systems
Evaluation with informational and navigational intents
Proceedings of the 21st international conference on World Wide Web
Minersoft: Software retrieval in grid and cloud computing infrastructures
ACM Transactions on Internet Technology (TOIT)
New approaches to diversity and novelty in recommender systems
FDIA'11 Proceedings of the Fourth BCS-IRSG conference on Future Directions in Information Access
A model for mining relevant and non-redundant information
Proceedings of the 27th Annual ACM Symposium on Applied Computing
Using Bayesian networks theory for aggregated search to XML retrieval
Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics
Top-k retrieval using facility location analysis
ECIR'12 Proceedings of the 34th European conference on Advances in Information Retrieval
On optimality-ratio and coverage in ranking of joined search results
Distributed and Parallel Databases
Privacy-aware image classification and search
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Diversity by proportionality: an election-based approach to search result diversification
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Explicit relevance models in intent-oriented information retrieval diversification
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Evaluating aggregated search pages
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Adaptive diversification of recommendation results via latent factor portfolio
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Personalized diversification of search results
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Combining implicit and explicit topic representations for result diversification
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Using preference judgments for novel document retrieval
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Advances on the development of evaluation measures
SIGIR '12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval
Diversification for multi-domain result sets
ICWE'12 Proceedings of the 12th international conference on Web Engineering
Possibilistic model for aggregated search in XML documents
International Journal of Intelligent Information and Database Systems
Coverage-based search result diversification
Information Retrieval
Reranking web search results for diversity
Information Retrieval
On the role of novelty for search result diversification
Information Retrieval
Constructing test collections by inferring document relevance via extracted relevant information
Proceedings of the 21st ACM international conference on Information and knowledge management
Social book search: comparing topical relevance judgements and book suggestions for evaluation
Proceedings of the 21st ACM international conference on Information and knowledge management
A comparative study of heterogeneous item recommendations in social systems
Information Sciences: an International Journal
Using semi-structured data for assessing research paper similarity
Information Sciences: an International Journal
Comprehension-based result snippets
Proceedings of the 21st ACM international conference on Information and knowledge management
A comprehensive analysis of parameter settings for novelty-biased cumulative gain
Proceedings of the 21st ACM international conference on Information and knowledge management
Contextual evaluation of query reformulations in a search session by user simulation
Proceedings of the 21st ACM international conference on Information and knowledge management
MOUNA: mining opinions to unveil neglected arguments
Proceedings of the 21st ACM international conference on Information and knowledge management
Measuring the coverage and redundancy of information search services on e-commerce platforms
Electronic Commerce Research and Applications
Effects of spam removal on search engine efficiency and effectiveness
Proceedings of the Seventeenth Australasian Document Computing Symposium
A study on novelty evaluation in biomedical information retrieval
SPIRE'12 Proceedings of the 19th international conference on String Processing and Information Retrieval
Diversifying user comments on news articles
WISE'12 Proceedings of the 13th international conference on Web Information Systems Engineering
Evaluating web archive search systems
WISE'12 Proceedings of the 13th international conference on Web Information Systems Engineering
mNIR: diversifying search results based on a mixture of novelty, intention and relevance
WISE'12 Proceedings of the 13th international conference on Web Information Systems Engineering
DisC diversity: result diversification based on dissimilarity and coverage
Proceedings of the VLDB Endowment
Modelling efficient novelty-based search result diversification in metric spaces
Journal of Discrete Algorithms
Proceedings of the sixth ACM international conference on Web search and data mining
SkyDiver: a framework for skyline diversification
Proceedings of the 16th International Conference on Extending Database Technology
Mining Divergent Opinion Trust Networks through Latent Dirichlet Allocation
ASONAM '12 Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012)
Using intent information to model user behavior in diversified search
ECIR'13 Proceedings of the 35th European conference on Advances in Information Retrieval
Reducing information redundancy in search results
Proceedings of the 28th Annual ACM Symposium on Applied Computing
Exploiting relevance, coverage, and novelty for query-focused multi-document summarization
Knowledge-Based Systems
User model-based metrics for offline query suggestion evaluation
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Sentiment diversification with different biases
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
A mutual information-based framework for the analysis of information retrieval systems
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
A general evaluation measure for document organization tasks
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Toward whole-session relevance: exploring intrinsic diversity in web search
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Search result diversification in resource selection for federated search
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Preference based evaluation measures for novelty and diversity
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Term level search result diversification
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Exploring semi-automatic nugget extraction for Japanese one click access evaluation
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Boosting novelty for biomedical information retrieval through probabilistic latent semantic analysis
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Informational friend recommendation in social media
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
Interactive exploratory search for multi page search results
Proceedings of the 22nd international conference on World Wide Web
Diversified recommendation on graphs: pitfalls, measures, and algorithms
Proceedings of the 22nd international conference on World Wide Web
Proceedings of the 22nd international conference on World Wide Web
Selecting effective expansion terms for diversity
Proceedings of the 10th Conference on Open Research Areas in Information Retrieval
Exploiting the diversity of user preferences for recommendation
Proceedings of the 10th Conference on Open Research Areas in Information Retrieval
On the reliability and intuitiveness of aggregated search metrics
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Evaluating aggregated search using interleaving
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Spatial search for K diverse-near neighbors
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Beyond search: Retrieving complete tuples from a text-database
Information Systems Frontiers
Diversified query expansion using conceptnet
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Learning to rank query suggestions for adhoc and diversity search
Information Retrieval
Mining subtopics from different aspects for diversifying search results
Information Retrieval
Increasing evaluation sensitivity to diversity
Information Retrieval
The water filling model and the cube test: multi-dimensional evaluation for professional search
Proceedings of the 22nd ACM international conference on Conference on information & knowledge management
Aggregated search: A new information retrieval paradigm
ACM Computing Surveys (CSUR)
POIKILO: a tool for evaluating the results of diversification models and algorithms
Proceedings of the VLDB Endowment
Geographic aspects of tie strength and value of information in social networking
Proceedings of the 6th ACM SIGSPATIAL International Workshop on Location-Based Social Networks
Diversely enumerating system-level architectures
Proceedings of the Eleventh ACM International Conference on Embedded Software
Latent dirichlet allocation based diversified retrieval for e-commerce search
Proceedings of the 7th ACM international conference on Web search and data mining
The notion of diversity in graphical entity summarisation on semantic knowledge graphs
Journal of Intelligent Information Systems
Composite retrieval of heterogeneous web search
Proceedings of the 23rd international conference on World wide web
Contextual and dimensional relevance judgments for reusable SERP-level evaluation
Proceedings of the 23rd international conference on World wide web
The neglected user in music information retrieval research
Journal of Intelligent Information Systems
Hi-index | 0.02 |
Evaluation measures act as objective functions to be optimized by information retrieval systems. Such objective functions must accurately reflect user requirements, particularly when tuning IR systems and learning ranking functions. Ambiguity in queries and redundancy in retrieved documents are poorly reflected by current evaluation measures. In this paper, we present a framework for evaluation that systematically rewards novelty and diversity. We develop this framework into a specific evaluation measure, based on cumulative gain. We demonstrate the feasibility of our approach using a test collection based on the TREC question answering track.