PROMISE retreat report prospects and opportunities for information access evaluation

  • Authors:

  • Maristella Agosti;Richard Berendsen;Toine Bogers;Martin Braschler;Paul Buitelaar;Khalid Choukri;Giorgio Maria Di Nunzio;Nicola Ferro;Pamela Forner;Allan Hanbury;Karin Friberg Heppin;Preben Hansen;Anni Järvelin;Birger Larsen;Mihai Lupu;Ivano Masiero;Henning Müller;Simone Peruzzo;Vivien Petras;Florina Piroi;Maarten de Rijke;Giuseppe Santucci;Gianmaria Silvello;Elaine Toms;Nicola Ferro;Richard Berendsen;Allan Hanbury;Mihai Lupu;Vivien Petras;Maarten de Rijke;Gianmaria Silvello

  • Affiliations:

  • University of Padua, Italy;University of Amsterdam, The Netherlands;Royal School of Library and Information Science, Denmark;Zurich University of Applied Sciences, Switzerland;National University of Ireland, Galway Ireland;Evaluations and Language resources Distribution Agency (ELDA), France;University of Padua, Italy;University of Padua, Italy;Centre for the Evaluation of Language and Communication Technologies (CELCT), Italy;Vienna University of Technology, Austria;University of Gothenburg, Sweden;Swedish Institute of Computer Science, Sweden;Swedish Institute of Computer Science, Sweden;Royal School of Library and Information Science, Denmark;Vienna University of Technology, Austria;University of Padua, Italy;University of Applied Sciences Western Switzerland (HES-SO), Switzerland;University of Padua, Italy;Humboldt University Berlin, Germany;Vienna University of Technology, Austria;University of Amsterdam, The Netherlands;Sapienza, University of Rome, Italy;University of Padua, Italy;University of Sheffield, United Kingdom;University of Padua, Italy;University of Amsterdam, The Netherlands;Vienna University of Technology, Austria;Vienna University of Technology, Austria;Humboldt University Berlin, Germany;University of Amsterdam, The Netherlands;University of Padua, Italy

  • Venue:
  • ACM SIGIR Forum
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

The PROMISE network of excellence organized a two-days brainstorming workshop on 30th and 31st May 2012 in Padua, Italy, to discuss and envisage future directions and perspectives for the evaluation of information access and retrieval systems in multiple languages and multiple media. This document reports on the outcomes of this event and provides details about the six envisaged research lines: search applications; contextual evaluation; challenges in test collection design and exploitation; component-based evaluation; ongoing evaluation; and signal-aware evaluation. The ultimate goal of the PROMISE retreat is to stimulate and involve the research community along these research lines and to provide funding agencies with effective and scientifically sound ideas for coordinating and supporting information access research.